Keywords

1 Introduction

Unmanned Aerial Vehicles (UAV) are applied on several applications like mapping, journalism, transport, rescue military applications and environments where a human cannot access [1]. Kendoul [2] classified UAVs into five categories according to the size and payload: Full-scale, Medium-scale, Small-scale, Mini, and Micro air vehicles (MAVs).

Most of the commercial MAVs depend on the skill of the pilot, the robustness of the communication system and sensors on board the vehicle. Sometimes there is difficult to handle the device due to loss of visibility or because the global position system (GPS) is not available. Autonomous system is an alternative for solving this issue. The autonomous systems include motion planning [3, 4], path tracking, obstacle avoidance, target detection and other areas [5, 6]. These systems require sensing, state estimation, perception, and knowledge of the situation. The perception is used to detect and avoid obstacles in real time, recognize and tracking objects and environmental mapping [7].

Our system works in a low-cost UAV with a monocular camera, uses a perception method based on feature points for obstacle detection and proportional control for obstacle avoidance, and no depends on other sensors.

2 Related Works

Research groups on robotics have proposed different techniques for obstacle avoidance, based on sensors like LIDAR [8,9,10] and Kinect [11, 12] that show robustness to identify obstacles, but these sensors involves an additional cost for the any UAV. There are several vision systems based on optical flow like [13], where authors propose a system for controlling an ultra-light airplanes with translatory optic flow that avoid obstacles and keep distance from ground and ceiling. Others approach are autonomous collision avoidance systems for navigation within houses or indoor environments using optical flow, micro-sensors and neural networks [14, 15]. A simulation of a navigation system with optical flow for rotary-wing UAV to avoid lateral and frontal collisions for a 3D urban environment is presented in [16]. Some of Optical flow problems is that the method cannot detect frontal obstacles [17].

In spite of the use of Scale Invariant Feature Transform (SIFT), to recognize collisions by analyzing the change in scale and location between two images [18, 19], is stable for some works but is not recommended due to the low speed. We use Speeded-Up Robust Features (SURF) to detect obstacles. There are several application of SURF like face detection [20], target tracking [21, 22], simple visual navigation [23], and some works with UAVs. One of these is [24] that uses a bang-bang control. Our work propose a real time known obstacle detection algorithm based on feature points, and an offline modeling of the MAV for designing a controller for fixed obstacle avoidance in unknown environment.

3 Our Approach

3.1 Obstacle Detection

We use two images, one is located in a database that contains obstacles and other is captured with the onboard camera. In order to find correspondence between these images, feature point detection, description and matching are used. Additionally we calculate the obstacle area and mass center to be used as target in the controller.

Feature Point

For feature point detection, there are several works in the literature [25, 26, 27], but Oriented FAST and Rotated BRIEF (ORB) [28], Fast Retina Keypoint (FREAK) [29], Binary Robust Invariant Scalable Keypoints (BRISK) [30], Scale Invariant Feature Transform [31], and Speeded Up Robust Feature (SURF) [32], are widely used algorithms [33, 34, 35, 36]. In our proposal, we use SURF because its the computational cost is lower without reducing robustness [37].

The algorithm SURF uses a basic Hessian matrix approximation that has a high performance regarding the accuracy [38] and integral images for reducing computational time [39]. The scale-space function uses an image pyramid smoothing with a Gaussian Filter for finding edges and blobs on different scales, and sub-sampling to obtain the next higher level of the pyramid. The feature point location is estimated with a combination of hessian matrix and scale-space function. SURF descriptor determines the distribution of the pixel intensity within a neighbour region for each detected feature point. The method used a Haar wavelet to decrease the computation time and increase the robustness, Haar wavelets are simple filters that determine the gradient at image, these filters are considerate how block based methods to calculate a directional derivatives of the image intensity [40].

There is necessary to find correspondence between the image from the database and the image captured with the UAV. We compare the feature points with the same type of contrast, achieving a lower computational cost without reducing the descriptor performance. Random Sample Consensus (RANSAC) [41, 42] discards the set of the pairs points out of the model, Fig. 1.

Fig. 1.
figure 1

Matching between image from database and real-time image at UAV

Full size image

Obstacle area and mass center

The obstacle area is inversely proportional to the distance between the obstacle and the UAV, however the camera perspective warps the obstacle geometry. There is required to compensate the perspective warping of the obstacle using a geometry transform. The affine transformation is widely used for motion compensation in [43,44,45]. This transformation is mathematically expressed as:

$$ H = \left[ {\begin{array}{*{20}c} {s\,\cos \left( \phi \right)} & { - s\,\sin \left( \phi \right)} & {t_{y} } \\ {s\,\sin \left( \phi \right)} & {s\,\cos \left( \phi \right)} & {t_{x} } \\ 0 & 0 & 1 \\ \end{array} } \right] $$
(1)

where \( \phi \) is rotation roll, \( s \) is the scale, \( t_{x} \) and \( t_{y} \) are the translation in x and y between the current and the last frame. We use the affine transformation to compensate the warping \( H \). Additionally, we calculate the area defined by the compensated vertex of the image from UAV. The fact that the rectangle is a regular figure, we estimate the mass center by the average value of the x and y coordinates of each vertex.

3.2 Obstacle Avoidance

The platform used in the experimentation is the Bebop Drone 1. The control system of the MAV manipulates four different control action: pitch, roll, yaw, and altitude. In our model we use control actions as input (Fig. 2a) and velocities as outputs (Fig. 2b). We proposed two motions in the plane x and y for the avoid system, in the x-axis the motion is uniform, i.e. the linear speed x is constant. The motion in the y-axis depends of the obstacle location, so the control law will be applied on this axis.

Fig. 2.
figure 2

System behavior to different input values (a) Speed input, (b) speed-output in x-axis, y-axis and z-axis

Full size image

Based on the low level control system of Bebop and the acquired dataset, we have considered that the mathematical relation between angles and motion in axis and can be represented by a static non linear model combined with a dynamic linear model. The models for each motion axis can be decoupled and define by the correlations: the motion in y-axis can be controlled by roll, and the motion in x-axis can be controlled by pitch. There is necessary to estimate the mathematical model between roll control and the linear speed in the y-axis. The non linear part of the model can be estimated as a polynomial regression for the angular inputs of angles and the stationary speed values in the axis. Nevertheless, these speed values trends to saturation levels. We estimated the motion model in y-axis that depends only on the roll input the mathematical short model:

$$ G\left( s \right) = \frac{K}{{T_{s} + 1}} $$
(2)

Controller design

Our consideration of controller design is which after detecting the obstacle the UAV should recovery the path. Also, we must define if the obstacle is on the trajectory, the position error, and the area of the obstacle. For bebop drone we capture 640 × 480 pixels for each image of visual field, where the obstacles are defined by the mass center as \( M_{c} \left( {y,z} \right) \) in the visual field image. The position error \( e\left( t \right) \) is the difference between the obstacle mass center and the center \( C\left( {y,z} \right) \) of the image (visual field): \( e\left( t \right) = C\left( {y,z} \right) - M_{c} \left( {y,z} \right) \).

If \( e\left( t \right) \) the obstacle is located in the right site of the drone otherwise the obstacle is in the left side. Maximum and minimum values of error are −320 and 320 for y-axis.

The obtained area is proportional to the distance between the onboard camera and the obstacle. In the Fig. 3 there is graphically explained the relation between the image plane and the obstacle length: \( \frac{{L_{1} }}{{IP_{1} }} = \frac{{L_{1} }}{{IP_{1} }} \) and \( \frac{{L_{2} }}{{IP_{2} }} = \frac{{L_{2} }}{{IP_{2} }} \).

Fig. 3.
figure 3

Relation between the area and distance.

Full size image

If \( L_{P2} > L_{P1} \), the image plane \( I_{P2} \) increase otherwise \( I_{P2} \) decrease. The relation between image plane and the distance between UAV and obstacle is \( \frac{{I_{P1} }}{{I_{P1} }} = \frac{{d_{1} }}{{d_{2} }} \).

Most of control systems are focused on reducing to zero the error between the output and set point. However, for the avoidance system, the controller increases the difference between the obstacle mass center and the visual field center. If the position error is low, i.e. the obstacle is close to the path center, the control system must send a higher speed signal to the motor controllers in order to keep distance respect to obstacles. In the other hand, when the position error is high, the vehicle is far from the path center reducing the collision probability. The control law depends on the location of the obstacle, is positive when the error is \( e\left( t \right) < 0 \) and negative when \( e\left( t \right) > 0 \). This means that the controller output \( u\left( t \right) \) is inversely proportional to the error \( e\left( t \right) \). We obtained a control law with a proportional gain \( K_{p} \) and a bias \( P \) defined as:

$$ u\left( t \right) = K_{p} *e\left( t \right) + P $$
(3)

Our algorithm starts when the obstacle area is greater than limit area, this means that the vehicle is closer to the obstacle. The limit area value is experimentally obtained defining the dimensions of obstacles in pixels at a specific distance. The output of the control law moves the vehicle away from the center of the obstacle avoiding it. If the error is greater than zero, the UAV moves to the left side, otherwise, the vehicle moves to the right side. The average speed obtained from the control signal is used for recovering the original path.

4 Results and Discussion

We performed an experimental comparison between our autonomous algorithm, the algorithm proposed by [24], and the tele-operation of two persons with different experience levels, using the same UAV.

The metrics of evaluation are: time to complete the path, maximum speed, minimum speed, average speed, distance and battery.

Table 1 presents the average values of the successfully flights for one fixed and two mobile obstacles, and the Table 2 a resume of them. The normalized values were obtained from the official Android App of Parrot.

Table 1. Results of one fixed and two mobile obstacles
Full size table
Table 2. Results of flights with different controls
Full size table

Experimental results showed that the autonomous system has lower ratio of battery usage and travelled distance, due to the system has not high variations of speed. The tele-operators used the less time to complete the path, for the ease of avoiding one obstacle, but if we compared with bang-bang system the time is similar.

Based on the experimental results, our proposal has a better performance than others controllers because the travelled distance and the time are lower, and additionally the stable speed allows successfully flights. In spite of that use of battery and the average time required to complete the path is lower for teleoperators, the number of unsuccessful flights ratio is higher. When the number of obstacles is higher the performance of human tele-operators decrease, as consequence of fatigue, in the other hand our proposal keeps this performance. Unlike the Bang-Bang controller, our proposal includes a path recovery system in order to return to the original trajectory. The importance of the path recovery system is evident when increases the number of obstacles, for example: for two or more obstacles, the bang- bang controller avoids the first obstacle and loses the path for the next obstacles, ending in other location. Our approach finishes in the correct goal.

5 Conclusions and Future Work

On this paper, we experimentally tested the optimal and robust performance of our system including obstacle detection and avoidance. Flexibility and energy efficiency are important features for autonomous navigation of UAVs. In our approach, flexibility is given by the effectiveness responding to unspecified number of fixed and mobile obstacles in unknown positions. SURF obtains matching between the image from the database and captured frame without incrementing the computational cost. Our proportional controller for obstacle avoidance between the start and goal point is optimal, faster and with higher performance than a bang-bang controller, human controllers with and without experience.