Research on Fusion of Multiple Positioning Algorithms Based on Visual Indoor Positioning

Abstract

Due to the limited penetration ability of GPS signal to buildings, indoor high-precision positioning combined with a variety of technologies has been paid more and more attention by researchers. Based on the traditional indoor positioning technology, a new indoor positioning method is proposed in this paper, which combines vision and inertial sensor. In this paper, we will first independently evaluate the quality of inertial positioning and visual positioning results, and then integrate them with complementary advantages to achieve the effect of high-precision positioning.

This work is supported by the National Natural Science Foundation of China (61771186), University Nursing Program for Young Scholars with Creative Talents in Heilongjiang Province (UNPYSCT-2017125), Distinguished Young Scholars Fund of Heilongjiang University, and postdoctoral Research Foundation of Heilongjiang Province (LBH-Q15121).

1 Introduction

In recent years, due to the continuous development of wireless communication networks and wireless sensor networks, people have higher and higher requirements for location services in complex indoor environments. Researchers have proposed a series of indoor location technology, known as WI-FI location, Radio Frequency Identification (RFID) location, Ultra Wideband (UWB) location and so on. In comparison, various technologies have both advantages and disadvantages.

At present, the focus of high precision indoor positioning research is to combine the advantages of different technologies. [1] describes the HiMLoc solution, which focuses on the advantages of Pedestrian Dead Reckoning (PDR) and WIFI fingerprint technology, that is, the solution only relies on the parameters of some buildings, as well as the accelerometer, WIFI card and compass in smart phones for positioning. However, the inertial sensor is unable to accurately extract the corresponding motion trajectory due to the constant checking of the mobile phone in the room. What is described in [2] is to evaluate the effectiveness of PDR in the corresponding camera through the change rate of pixels, which can effectively improve the positioning accuracy of PDR. In [3], Ashish Gupa and Alper Yilmaz showed that their proposed indoor positioning method is to install a plane building information model and multi-directional sensor suite in the mobile phone. The above research improves the accuracy of indoor positioning system to a certain extent.

Visual indoor positioning and PDR indoor positioning are two common indoor positioning methods. [4,5,6] describe the use of visual indoor positioning, which has the advantages of low power consumption and low cost. However, some disadvantages of the smart phone itself, such as the resolution of the camera itself, as well as external lighting, will affect its imaging. In 2016, Google came up with Tango and launched the first Tango smartphone with Lenovo. Tango is known for its regional learning, a feature that, when combined with a smartphone, allows it to capture and remember architectural features of interior Spaces quickly and efficiently, such as corners, walls and bumps. Tango has its own coordinate system and its own method of extracting feature points, which makes Tango’s positioning system more accurate. But in indoor the place with darker light, the accuracy of this certain bit method is about to discount greatly.

By contrast, Tango visual positioning is highly reliable in indoor positioning accuracy, except that it has certain shortcomings in light. Therefore, we need to use other indoor positioning techniques to make up for Tango’s shortcomings. The smart phone inertial sensor described in [8] is similar to Tango visual positioning in that it USES PDR and Tango visual positioning, because both of them are relative positioning technologies that estimate walking direction and walking distance from known positions. [9] describes the main problems existing in PDR: its positioning accuracy will gradually decrease with the passage of time, and the error will accumulate more and more.

This paper presents an indoor positioning algorithm based on the combination of visual positioning and inertial sensor. The algorithm combines the advantages of the two methods and achieves better precision and robustness. The algorithm will be described in the next section. The third part is the experimental results and the fourth part is the conclusion.

2 System Description

The system block diagram of the indoor positioning system proposed in this paper is shown in Fig. 1. Tango is used for visual location measurement. The inertial sensor is located using the accelerometer and gyroscope inside the smartphone. These two parts of the system do not affect each other and work independently. Then the accuracy of the positioning results will be evaluated, and the loose coupling method is taken to achieve the fusion.

Fig. 1.

System architecture

2.1 Tango Visual Interior Location Algorithm

The cyclic process of Tango device positioning is screening, identifying feature points, matching feature points, filtering error matches, and coordinate transformation. In Tango positioning, the motion and acceleration direction of the smartphone are measured by the accelerometer and gyroscope in the device, and the measured sensor data are fused to solve the accumulated errors in the motion through regional learning, so as to achieve the effect of three-dimensional motion tracking. The coordinates Tango uses for its location are in a custom virtual framework, so the coordinate transformations must be made to integrate with PDR. The expression to convert the coordinates in the PDR frame into the coordinates in the Tango visual frame is:

$$ x_{T} = \left( {x_{P} - x_{0} } \right) \times \cos \theta - \left( {y_{P} - y_{0} } \right) \times \sin \theta $$

(1)

$$ y_{T} = \left( {y_{P} - y_{0} } \right) \times \cos \theta + \left( {x_{P} - x_{0} } \right) \times \sin \theta $$

(2)

Instead, the expression for converting coordinates in Tango visual frames to coordinates in PDR frames is will be as (3) and (4).

$$ x_{P} = x_{0} + y_{T} \times \sin \theta + x_{T} \times \cos \theta $$

(3)

$$ y_{P} = y_{0} + y_{T} \times \cos \theta - x_{T} \times \sin \theta $$

(4)

where $\left( {x_{0} ,y_{0} } \right)$ is the origin coordinate of Tango visual frame in PDR positioning frame. $\theta$ is the angle between Tango visual frame $y$ axis. $\left( {x_{P} ,y_{P} } \right)$ is the coordinate in PDR positioning frame. $\left( {x_{T} ,y_{T} } \right)$ is the coordinate in Tango visual framework.

The sampling frequency of Tango is $100\,{\text{Hz}}$, and the output is a continuous sample value. We need to find the zero velocity point from the coordinates of the output value, and use this point to estimate the coordinates of step size and heading. Where, the step size and heading can be calculated by (5) and (6):

$$ L = \left\| {P_{{K_{N} }} - P_{{K_{N - 1} }} } \right\| $$

(5)

$$ \varPsi = angle\left( {\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {P}_{{K_{N} }} - \overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {P}_{{K_{N - 1} }} } \right) $$

(6)

2.2 PDR Localization Algorithm

PDR is a relative, cumulative position of the navigation technology. Namely, starting from a known position, the displacement generated by the target’s motion trajectory is added. The displacement calculation can be given in the form of a change in Cartesian coordinates or a change in heading.

According to the inverted pendulum model described in [11], the vertical distance can be converted into the horizontal step length. The inverted pendulum model is shown in Fig. 2.

Fig. 2.

Inverted pendulum model

where $L$ is the radius of the model, indicating the target step; $h$ is the vertical displacement, and the time interval of this displacement is from the moment of landing on the heel to the moment of standing firm. $H$ can be obtained by (7). Step length $D$ can also be obtained through Pythagorean Theorem with the first half $D_{1}$ and the last half $D_{2}$ as (8):

$$ h = \mathop {\iint }\nolimits a\left( t \right)\text{d}t $$

(7)

$$ D = D_{1} + D_{2} = \sqrt {2Lh_{1} - h_{1}^{2} } + \sqrt {2Lh_{2} - h_{2}^{2} } $$

(8)

Using the angular information and quaternion algorithm in the 6-axis inertial sensor, we can transform the coordinates in the pedestrian coordinate system into the coordinates in the navigation coordinate system by using the coordinate trans-formation matrix as in (9).

$$ c_{b}^{n} = \left[ {\begin{array}{*{20}c} {T_{11} } & {T_{12} } & {T_{13} } \\ {T_{21} } & {T_{22} } & {T_{23} } \\ {T_{31} } & {T_{32} } & {T_{33} } \\ \end{array} } \right] $$

(9)

The direction of pedestrian can be calculated from (10):

$$ \varPsi = \tan^{ - 1} \left( {\frac{{T_{12} }}{{T_{22} }}} \right) $$

(10)

where $\varPsi$ is defined between $0$ and $360^{ \circ }$.

2.3 Fusion Algorithm of Tango and PDR

Compared with PDR localization, Tango can achieve relatively accurate indoor positioning result with a good lighting condition. The lighting sensitivity, however, may be the weakness for Tango, because if the lighting conditions change, such as too much reflection on the indoor floor or too much white on the surrounding walls, it will affect Tango’s positioning effect. And PDR positioning although will over time to produce the error affecting the positioning accuracy of the system, but the IMU (Inertial Measurement Unit) in each point estimation error is reliable, so the IMU is used to determine the effectiveness of the Tango dots: if the dots in the range of error, the output is valid, otherwise is invalid. The error range $\varepsilon$ is shown in Fig. 3.

Fig. 3.

Judging the validity of Tango output

The validity of step size and heading observed in Tango can be expressed as (11) and (12):

$$ \begin{aligned} \left\| {\overset{\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {L}^{V} } \right\| \in \left( {\left\| {\overset{\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {L}^{I} } \right\| - \varepsilon_{1} ,\left\| {\overset{\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {L}^{I} } \right\| + \varepsilon_{1} } \right)\\ \end{aligned} $$

(11)

$$ \left\| {\overset{\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {\theta }^{V} } \right\| \in \left( {\left\| {\overset{\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {\theta }^{I} } \right\| - \varepsilon_{2} ,\left\| {\overset{\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {\theta }^{I} } \right\| + \varepsilon_{2} } \right) $$

(12)

$L^{V}$ and $L^{I}$ are the step size estimation of tango visual positioning and PDR positioning, respectively. $\theta^{V}$ and $\theta^{I}$ are the course observed by tango visual positioning system and PDR positioning system respectively. Kalman filter is part of inertial navigation system. In this paper, Kalman filter and heading are used for fusion: once the inertial navigation system detects the zero velocity value point, it will trigger the Kalman filter to change the vertical velocity to zero. Then the optimal estimation is obtained from the measured value of the current state:

$$ X\left( {k |k} \right) = X\left( {k |k - 1} \right) + Kg\left( k \right) \times \left( {Z\left( k \right) - HX\left( {k |k - 1} \right)} \right) $$

(13)

(13) above can be expanded as (14):

$$ X\left( {k |k} \right) \approx X\left( {k |k - 1} \right) + Kg\left( k \right) \times \left( {Z\left( {k - 1} \right) - HX\left( {k - 1 |k - 2} \right)} \right) $$

(14)

where $Z\left( {k - 1} \right)$ is the course observation detected by Tango at the previous moment. $X\left( {k - 1 |k - 2} \right)$ is the course information estimated by PDR positioning system in the previous moment. The previous non-zero vertical velocity is fed back to the Kalman filter as compensation so that the cumulative error can be eliminated.

3 Performance Analysis

The accuracy of the proposed method is evaluated and analyzed by conducting field positioning experiments. The experiments are performed by Tango smart phone jointly launched by Google and Lenovo: Lenovo PHAB 2 Pro. The location was located on the 7th floor of the physical experiment building of Heilongjiang University. We will use the verification point to represent the ground truth value, carry out error analysis, and compare with the traditional positioning method.

3.1 Tango Visual Positioning Evaluation

Figure 4 shows the results of Tango. The red dots indicate that we are at the preset checkpoint, the Tango output trajectory is blue, and the red lines indicate the walls of the floor plan of the experimental building. Tango’s output jumps, as shown in the black circle, because the floor reflects light at that point.

Fig. 4.

Results of Tango positioning experiments (Color figure online)

3.2 PDR Positioning Evaluation

Figure 5 shows the results of PDR positioning. The results of PDR positioning deviated greatly from the areas we specified. As mentioned above, the error gradually increases with time, which leads to the black point in Fig. 5 moving from $\left( { - 2.6,1.1} \right)$ to $\left( { - 0.8,3.8} \right)$.

Fig. 5.

Result of PDR positioning experiment

3.3 Analysis of Fusion Positioning System

Figure 6 intuitively shows that the output trajectory of Tango and PDR fusion positioning system is very similar to that of Tango visual positioning system. However, it can be seen from Fig. 6 that when the light ray changes significantly, Tango displacement is inconsistent with the normal displacement, especially with the merging system.

Fig. 6.

Result of Tango and PDR fusion positioning experiment

The accuracy of the three positioning systems are compared, and the median, mean, root-mean-square and three-fourths of the positioning errors of all the markers are calculated. As it can be seen from Table 1, the accuracy of the three positioning systems is higher than that of Tango visual positioning system and higher than that of PDR positioning system from high to low. The median error of all markers was similar. The mean error of all markers in the fusion system is lower than that of Tango positioning system.

Table 1. Comparison of positioning errors

Figure 7 shows the error distribution of the two typical positioning methods. The X-axis is the positioning error and the Y-axis is the cumulative probability. The significance of any point on the curve is that 90% of the positioning error is below (assuming $y$ is less than $0.9$) $x$ meters. When $y = 1$, its corresponding X-axis value is the maximum error of this experiment. On the contrary, when $y = 0$, the corresponding X-axis value is the minimum error of this experiment.

Fig. 7.

Coordinate structure and error analysis on (a) X-axis and (b) Y-axis

Moreover, the error accumulating percentage of the two simulated positioning methods is shown in Fig. 8, which indicates that the error of Tango positioning system is 80% the same as that of fusion positioning system, but the error of Tango will increase greatly once the light changes. In conclusion, the accuracy of fusion positioning system is higher.

Fig. 8.

Accumulating error percentage comparison

4 Conclusion

In this paper, a kind of indoor positioning system is proposed, which integrates the visual positioning system and inertia-based positioning system in a loosely coupled architecture. The measurements obtained by Tango are evaluated using the PDR output. If the step size given by Tango is reliable, it will be directly used in the fusion system; otherwise, the step size will be derived from the inverted pendulum model. The fusion system also uses a Kalman filter for course fusion. Data from the inertial sensor is used for prediction, and Tango provides measurements. Experimental results show that, compared with the traditional PDR or Tango positioning method, we achieve a more accurate indoor positioning system.