An Indoor Navigation Service Robot System Based on Vibration Tactile Feedback

Abstract

In order to help the visually impaired and those who need the help of guidance, an indoor navigation service robot system based on vibration tactile feedback is proposed in this paper. Being different from the physical interaction typically used in existing navigation systems, the interaction in the proposed system is achieved by a wearable vibro-tactile bracelet device. Moreover, a human-guidance algorithm is presented based on the basic leader–follower control model. In the guidance, the user is free to decide the linear velocity, and the angular velocity is appropriately adjusted by the vibro-tactile feedback. The feedback is given in the form of the vibrational signal of the bracelet. In the whole process, the distance and orientation deviations between the human and the robot can be controlled in a certain scope, thus the system can ensure that the robot guides the user to the destination accurately. The tactile perception experiments show that the tactile indication method of the bracelet is effective. The trajectory following experiments on the flat ground and the adaptability test on the undulating terrain are performed to verify the feasibility of the system. The user acceptance tests show that the system is user-friendly and less intrusive. Finally, the anti-interference ability of the system is analyzed via experiments to verify the robustness of the system.

1 Introduction

Due to the lack of visual information, the visually impaired suffer from the limited scope of activities and increasing risk of walking danger. Similarly, when a normal person is in a complex and unknown indoor environment with limited field of view, how to accurately reach the destination becomes a difficult task. Trained guide dogs and canes can partially meet these navigation needs, thus they are widely used. However, they can only offer short-distance navigation and cannot tell people in detail about the environment around. Furthermore, the use of guide dogs is high-cost, vulnerable to environment and has many limitations in some indoor public places.

Guidance systems have been reported in a large body of literature. The authors in [1] developed a navigation system where the blind was equipped with a digital camera, a GPS receiver and a headset. The operator navigated the user toward a desired location by using audio communication. A system which used a head mounted camera to capture environment information and converted image information into sound output was proposed in [2]. However, the GPS localization is not available indoors; the audio messages may distort and mask the other useful sounds in the surrounding environments; the problem of information overload may occur in the auditory and visual channels; systems of this sort cannot work well in the noisy environments.

Some researchers transfer the method of hint to haptic stimuli. The authors proposed a vibro-tactile belt for guiding blind walkers. This belt enabled blind walkers to receive haptic directional instructions along complex paths [3]. The proposed indoor localization system in [4] provided surrounding environment information by vibro-tactile stimuli displayed to the chest and the shoulders of the user. Unfortunately, the belt and other similar interfaces must be calibrated for each body, so they are not very convenient to use. They may interfere with regular activities and bring a person uncomfortable feeling. Bhatlawande developed an electronic bracelet as an assistant tool when the user is walking with a cane [5]. The bracelet provided vibrations when an obstacle was close to the subject. The vibration magnitude was directly proportional to the obstacle distance. The bracelet can truly decrease the physical interference, while it did not give the user any directional information.

With the development of the mobile robot technology, many researchers have engaged in studying the service robot and human–robot cooperation. Mobile robots can assist people in some complex tasks, such as indoor and outdoor navigation [6, 7], search and rescue [8], or surveillance [9]. Some researchers apply the robot technology to design new guidance systems. A framework for the navigation and target tracking system for a mobile robot was proposed in [10]. The researchers in [11] provided an algorithm to maintain a distance between the robot and the object. It can keep the autonomous mobile robot at a safe distance from the object. Scheggi proposed a novel use of haptic feedback for human navigation with a mobile robot [12]. It was validated that the robot can steer the user to a desired target position by interacting with the user via a vibro-tactile bracelet. A new visuo-haptic interaction mechanism was applied to human–robot formation control [13]. The formation setup consisted of a human leader and multiple follower robots. A prototype assistive-guide robot named as eyeDog that provided the blind with autonomous vision-based navigation and laser-based obstacle avoidance capabilities was introduced in [14]. However, the user needed to adjust the walking speed according to the linear velocity of the robot.

Based on vibration tactile feedback, an indoor navigation service robot system consisting of a robot platform and a vibro-tactile bracelet device is proposed in this paper. The robot platform is based on a small tracked mobile robot which carries a backward facing RGB-D sensor. Unlike the physical interaction of the kinesthetic feedback (haptic feedback perceived by sensors in the muscles, joints, tendons), a vibro-tactile bracelet is designed as a tactile interface in this work. The motivations of using a vibro-tactile bracelet are more convenient, more portable, less obstructing and a wider range of motion. In the physical interaction of the kinesthetic approaches, the user is difficult to accomplish other physical tasks because his hands are occupied. It is also not convenient to extend the physical interaction to multiple users. However, in our proposed system, no physical interaction is required between the human and the robot. Moreover, owning to the use of wearable devices, the approach can be extended to other bodies for more applications. According to the basic leader–follower control model which is widely used in the multi-robot formation, a human-guidance algorithm is proposed to solve the human-guidance problem. This control policy can adjust the linear velocity of the robot according to the walking speed of the user.

In the practical application, the human should always be able to freely select the pace. Nevertheless, a specific haptic feedback is sent to the user in order to adjust the heading according to the specific deviation from the desired path. Our goal is to maintain a predefined relative distance and orientation between the human and the robot. As shown in Fig. 1, the robot can guide the human to walk from the start to the end. Only by interacting with the user via a vibro-tactile bracelet, the proposed navigation system can help the user follow a predefined trajectory correctly and safely.

The rest of the paper is organized as follows. Section 2 contains an overview of the proposed system, including the robot platform and the vibro-tactile bracelet device. Section 3 proposes a human-guidance algorithm based on the leader–follower model. Section 4 describes the experimental results to show the system performance and user acceptance. Finally, conclusions and some future work are given in Sect. 5.

Fig. 1

Human-guidance from a start to an end

2 System Overview

2.1 Robot Platform

The robot platform consists of a small tracked mobile robot, an RGB-D sensor, a ZigBee wireless module and a tablet PC. The platform has a size of 275 mm $\times$ 280 mm $\times$ 400 mm, as shown in Fig. 2.

A small tracked mobile robot is chosen as the carrier of the platform for its strong flexibility, good adaptability to complex environment and easiness to be equipped with other devices. The robot is 265 mm $\times$ 235 mm $\times$ 150 mm in size and 2500 g in weight. With a maximal linear velocity of 1.2 m/s and a load capacity of 2 kg, the robot can meet the demand of the system.

The Kinect which is used as the RGB-D sensor of the system can provide 20 joint positions of human skeleton. Seven joints are chosen, i.e., Shoulder Center, Shoulder Right, Shoulder Left, Spine, Hip Center, Hip Right, and Hip Left to get the relative position of the user to the robot. When people are walking, these joints have smaller amplitude of swing. Figure 3 shows the distribution of 20 joints of the skeleton captured by the Kinect.

Fig. 2

The prototype of the robot platform

Fig. 3

Distribution of 20 joints of the skeleton captured by the Kinect

Fig. 4

The flowchart of systematic process

As shown in Fig. 4, after processing the skeleton data of the user captured by the Kinect, the PC sends the control command to the robot (MCU) and the vibration command to the ZigBee module 1 (MCU1) by serial ports. Then the MCU1 sends the command to the ZigBee module 2 (MCU2) which is connected with the vibration motors of the bracelet via the ZigBee communication. Based on these commands, both the robot and the user can adjust their movements accordingly.

2.2 Vibro-tactile Bracelet Device

The vibro-tactile interfaces have been widely investigated. Tactile vibratory sensitivity is influenced by the spatial location on the body, the distance between the stimulators, the frequency of stimulation, the age of the subject and so on. Some studies showed that the wrists of human perform best in feeling vibrations [15]. A bracelet is a good choice for the vibro-tactile feedback. Vibro-tactile displays are usually based on the flat coin motors which are akin to the vibrators used in mobile phones.

Since the warning information about the linear and angular velocity of the user is needed, a haptic bracelet equipped with four flat coin motors which are 10 mm in diameter and 3 mm in thickness is designed. The left, right, front and back motors are denoted as motor L, motor R, motor F and motor B respectively. These motors were placed into the internal surface of an elastic belt whose width is about 100 mm. The minimal distance between two stimuli to be differentiated on the forearm is about 35 mm [16]. In the designed bracelet, the distance between two adjacent motors is set as 80 mm. The ZigBee module 2 was hidden in a small box of 60 mm $\times$ 80 mm $\times$ 55 mm which also contains a 3.7 V battery to power the overall device. Figure 5a shows the worn vibro-tactile bracelet. The unfolded vibro-tactile bracelet is presented in Fig. 5b.

Fig. 5

The prototype of vibro-tactile bracelet device. a The worn vibro-tactile bracelet. b The unfolded vibro-tactile bracelet

The main goal of tactile displays is to provide information intuitively as easy as possible. To this end, we adopt a simple on/off mechanism rather than tuning the vibration parameters. The maximal sensitivity can be achieved around a frequency of 200–300 Hz [17]. When a motor is engaged, it vibrates at a frequency of 250  Hz. The Kinect has its limited field of view. An equation can be gotten from Fig. 6a as follows:

$$\begin{aligned} H=h+Ltan(21.5^{\circ }+\theta ) \end{aligned}$$

(1)

where H and h are the heights of the user and the Kinect respectively, L is the horizontal distance between them and its value is larger than 120 cm, $\theta$ is the inclination degree of the Kinect.

Fig. 6

The detectable field. a The vertical field of vision. b The horizontal field of vision, i.e. the distribution of the location of the user

The Kinect has an interactive distance range of 120–350 cm. When L, $\theta$, and h are set as 150 cm, $20{^{\circ }}$, and 40 cm respectively, the calculated value of H is 173 cm. This value ensures that most people can be tracked by the Kinect. The vibration tactile feedback policy is described as follows. The region MNQP in Fig. 6b is the detectable scope when O is the position of the Kinect. In terms of the judgements on distance and orientation, there are totally nine possible areas of the location of the user. The region MNQP is divided into nine parts. In MNQP, the region C which is the ideal area is set as a square of 20 cm $\times$ 20 cm. This kind of design is used to avoid excessive alteration between two opposite haptic stimuli. Other regions mean different deviations from the region C, for example, the region FR means the front and right side of the region C. Figure 6b shows the distribution of the location of the user. Corresponding to these regions, nine vibration modes are encoded in the form of the on/off mechanism of motor F, R, B, and L. The “1” in vibration code denotes on, while “0” denotes off. Table 1 shows the relationship between the vibration codes and the regions of user. For example, the user will feel the vibrations of motor B and motor L in the region BL. It means that the subject needs to turn right and can try to walk faster.

Table 1 The relationship of vibration code and region of user

3 Control Algorithms

3.1 Basic Leader–Follower Formation Control

In the multi-robot formation, the leader–follower control is widely applied in many application scenarios [18, 19]. In the formation, robot $R_{j }$ follows robot $R_{i}$ with a desired separation $l^{d}$ and desired relative bearing $\psi ^{d}$ as shown in Fig. 7a.The kinematics of a robot can be abstracted as a unicycle model:

$$\begin{aligned} \dot{x} = v\cos \theta , \dot{y} = v\sin \theta ,\dot{\theta } = \omega \end{aligned}$$

(2)

where we let ${{\varvec{x}}} = (\hbox {x}, \hbox {y}, \theta ) \in \, \hbox {SE(2)}$, v and $\omega$ are the linear and angular velocities of the robot respectively.

Fig. 7

a The basic leader–follower model. b The human-guidance model

The leader–follower system is transformed into a new set of coordinates where the state of the leader is treated as an exogenous input. The kinematic equations are given by:

$$\begin{aligned} \left\{ {\begin{array}{l} \dot{l}_{{ ij}}=v_j \cos \gamma _{ij} -v_i \cos \psi _{ij} +d\omega _j \sin \gamma _{ij} \\ \dot{\psi }_{ij} =\frac{1}{l}_{ij}\left( {v_i \sin \psi _{ij} -v_j \sin \gamma _{ij} +d\omega _j \cos \gamma _{ij} -l_{ij}w_i } \right) \\ \dot{\beta }_{ij} =\omega _i -\omega _j \\ \end{array}} \right. \end{aligned}$$

(3)

where $l_{ij}$ and $\psi _{ij}$ are the current relative distance and orientation of $R_{i}$ and $R_{j}$, $\gamma _{ij}= \theta _{i}+\psi _{ij }- \theta _{j}$, d is the offset to an off-axis reference point $P_{j }$ on the $R_{j}$. In order to avoid collisions between robots, the length of $l_{ij}$ is required to be longer than the length of d. The kinematic equations can be simplified as follows:

$$\begin{aligned} \left\{ {\begin{array}{l} \dot{z}_{ij} =G_1 u_j +G_2 u_i \\ \dot{\beta }_{ij} =\omega _i -\omega _j \\ \end{array}} \right. \end{aligned}$$

(4)

where $z_{ij}=[l_{ij}\; \psi _{ij}]^{T}$ is the output of the system, $u_{j}=[v_{j}\; \omega _{j}]^{T}$ is the input for $R_{j}, u_{i}=[v_{i}\; \omega _{i}]^{T}$ is the input for $R_{i}$, and

$$\begin{aligned} G_1 =\left[ {{\begin{array}{ll} {\cos \gamma }_{ij}&{}\quad {d \sin \gamma _{ij} } \\ {\frac{-\sin \gamma _{ij}}{l_{ij}}}&{} \quad {\frac{d \cos \gamma _{ij} }{l_{ij} }} \\ \end{array} }} \right] \;\; G_2 =\left[ {{\begin{array}{ll} {-\cos \psi _{ij}}&{} \quad 0 \\ {\frac{\sin \psi _{ij}}{l_{ij}}}&{} \quad {-1} \\ \end{array} }} \right] \end{aligned}$$

(5)

By applying input–output feedback linearization, the control velocities for the follower $R_{j}$ can then be written as:

$$\begin{aligned} u_j =G_1^{-1}\left( {q-G_2 u_i } \right) \end{aligned}$$

(6)

where q is an auxiliary control input given by:

$$\begin{aligned} q=\left( {{\begin{array}{l} {k_1\left( {l^d - l_{ij}} \right) } \\ {k_2\left( {\psi ^{d}-\psi _{ij}} \right) } \\ \end{array} }} \right) =k(z_{ij}^d -z_{ij} ) \end{aligned}$$

(7)

$k_{1}$ and $k_{2} > 0$ are the user-selected controller gains. Note that $G_{1}$ is always reversible as long as the value of d is larger than 0, which must be true in practice.

When the linear and angular velocities of the leader robot $R_{i}$ is $v_{i}$ and $\omega _{i}$, the control algorithm represented by (6) allows the robot $R_{j}$ to follow the robot $R_{i}$ with the desired distance $l^{d}$ and orientation $\psi ^{d}$ between them.

3.2 Human-Guidance Algorithm

With a growing interest in human–robot interaction, a growing number of studies have explored strategies in mixed human–robot formation control [20, 21]. In fact, recent studies have shown that human locomotion in goal-directed movements can be approximated by the motion of a nonholonomic system [22]. The nonholonomic constraints of walking human are similar to those described in wheeled robots. In particular, the segments of shoulder are proved to be the best reference frames. This observation indicates that humans mostly walk on the smooth paths and the direction of their body is tangent to the trajectories they follow. Then in the following part, some alteration will be applied in the basic leader–follower model to solve the human-guidance problem.

In the basic leader–follower model, the leader leads the formation and controls the motion of the follower. However, the robot in the proposed system leads the human to the destination on the predefined path, and its speed is adjusted by the user’s pace. In other words, the robot leads the user to walk along the trajectory, and the user leads the whole speed of the system. In our practical application as shown in Fig. 7b, the user $H_{j}$ is free to choose the linear velocity $v_{j}$ to ensure the autonomy of the system. However, in order to be guided by the leader robot $R_{i}$, the angular velocity of the user should be appropriately regulated by the vibro-tactile feedback. When the user $H_{j}$ adjusts the angular velocity $\omega _{j}$, it also means the heading $\theta _{i}$ is modified. The robot $R_{i}$ adjusts its linear velocity $v_{i}$ dynamically according to that of the user $H_{j}$, and the angular velocity $\omega _{i}$ of the robot $R_{i}$ depends on the predefined trajectory. The robot $R_{i}$ has enough environment perception capability and can plan the correct trajectory from the start to the end. The trajectory is considered smooth.

In the human-guidance, the control velocities of the system are the angular velocity of the human $\omega _{j}$ and the linear velocity of the robot $v_{i}$ which are given by [cf., (6)]:

$$\begin{aligned} \left[ {{\begin{array}{l} {\omega _j } \\ {v_i } \\ \end{array} }} \right] =F_1^{-1}\left( {Q-F_2 \left[ {{\begin{array}{l} {v_j } \\ {\omega _i } \\ \end{array} }} \right] } \right) \end{aligned}$$

(8)

and

$$\begin{aligned} F_1 =\left[ {{\begin{array}{ll} {\frac{d\cos \gamma _{ij} }{l_{ij} }}&{} \quad {\frac{\sin \gamma _{ij}}{l_{ij}}} \\ {d\sin \gamma _{ij} }&{} \quad {-\cos \gamma _{ij}} \\ \end{array} }} \right] \;\; F_2 =\left[ {{\begin{array}{ll} {\frac{-\sin \psi _{ij}}{l_{ij}}}&{} \quad {-1} \\ {\cos \psi _{ij}}&{} \quad 0 \\ \end{array} }} \right] \end{aligned}$$

(9)

where $l_{ij}$ and $\psi _{ij}$ are the current relative distance and orientation of the robot $R_{i}$ and the user $H_{j}$, $\gamma _{ij}= \theta _{i}+ \psi _{ij }- \theta _{j}$, $\theta _{i}$ and $\theta _{j}$ are the heading of $R_{i}$ and $H_{j}$ respectively, d is the offset to an off-axis reference point on the $H_{j}$, Q is an auxiliary control input similar to (7), which is defined as:

$$\begin{aligned} Q=\left( {{\begin{array}{l} {k_2\left( {\psi ^{d} -\psi _{ij}} \right) } \\ {k_1\left( {l^{d} -l_{ij}} \right) } \\ \end{array} }} \right) \end{aligned}$$

(10)

Note that $l^{d}$ and $\psi ^{d}$ should be reasonably chosen to make sure that the human can be tracked by the Kinect on the robot. In the practical application, the user can freely select the linear velocity. The linear velocity of the robot is adjusted according to the control policy represented by (8). When the linear velocity of the user is $v_{j}$ and the angular velocity of the robot is $\omega _{i}$, the control algorithm can provide the desired linear velocity of the robot. It is not problematic to drive a robot with velocity $v_{i}$. The angular velocity of the human $\omega _{j}$ is dynamically adjusted by the vibration tactile feedback policy (cf., Fig. 6 and Table 1 in Sect. 2.2). When the user perceives the vibration of the motor L or motor R, he or she should adjust the heading to the right or left and take the next step. It is true that the human can freely choose the pace. Since the real robot has strict limitations on its maximum linear velocity and some environments have speed limitations on the safe velocity of the robot, the motor F and motor B are provided as the auxiliary reminders to the user.

In this section, a cooperative navigation control policy is presented for human–robot teams. The human and the robot are modeled as first-order systems with velocity control inputs. Combining the vibration tactile feedback, the control algorithm represented by (8) allows the desired distance $l^{d}$ and orientation $\psi ^{d}$ to be kept between the human and the robot in the whole navigation process.

4 Experimental Results

4.1 Tactile Perception Experiment

It is necessary to test the efficiency of the haptic bracelet before using the system. Before the experiment, the testers should take five minutes to learn the correspondence between the vibration modes and the indication meanings. The bracelet has been tested on 10 healthy subjects whose ages range from 22 to 28. There are 45 tactile perception trials which consist of five repetitions of nine modes in a randomized order. Within a trial, the stimulus is kept until the subject is able to give an answer. After each 5 trials the subjects take a small break.

In order to reduce the vibration disturbance of the adjacent motors and avoid the aftereffect problem (see [23] and the references therein), the vibration is displayed in a way of alternation between the two stimuli in the combinational modes (FR, BR, BL, FL). Specifically, the alternation means a periodic vibrational pattern with a period of 0.5 s. In that case, the motors of the bracelet are activated with a square wave which has a duty cycle of 50%. Figure 8a shows the relationship between the recognition rates and the vibration modes which correspond to the nine regions in Table 1. The consuming time that the users recognize the given stimuli is presented in Fig. 8b.

Fig. 8

a The recognition rate of nine vibration modes. b The perception time of the given stimuli

From the results, the recognition rate of each mode is higher than 90%. The average recognition rate of all modes is 95.33%. Meanwhile, the recognition rates of single modes (C, F, R, B, L) are all higher than those of the combinational modes. Also the average perception time of the vibration signals is around 2 s. It is obvious from the experimental results that the tactile indication method is effective and useful. In order to get higher perception accuracy, subjects should carry on more practice. Therefore, the designed vibro-tactile bracelet can meet the requirements of the system.

Fig. 9

The sketch map of feasibility test

Fig. 10

The experimental scene on the flat ground

4.2 Feasibility Test

In order to test the effectiveness of the system, we choose an indoor environment in a building in our campus to perform the trajectory following experiments on the flat ground. The sketch map of feasibility test which includes an L-shaped path is shown in Fig. 9. An L-shaped path is selected as the robot trajectory. The L-shaped path includes two straight segments and one curved segment which are two kinds of the most common segments. Twelve volunteers take part in this experiment. Ten of them have participated in the tactile perception experiment before. None of them report any deficiencies in the perception abilities (including vision, hearing and touch). In preparing work of the trajectory following experiments, the subjects are blindfolded in a waiting room. After that, they are leaded to the indoor experimental scene. The whole walking environment is unknown to each subject, thus they have poor space knowledge with respect to the experiment environment. All subjects are instructed to move according to the haptic feedback, but no instructions are given about their linear velocities. The experimental scene is shown in Fig. 10. Without the help of others, the guidance task is to follow safely the L-shaped path shown in Fig. 9 from the start to the end. All the testers have no idea of the path in such an environment, and they follow the path for only once. The whole guidance procedure is recorded by video. Owing to the actuation time of the motors of the robot, the control inputs of the human and the robot are computed every 0.1 s. Regarding the formation parameters, we set $l^{d} = 2.0\;\hbox {m}$, $\psi ^{d}=\uppi$, $k_{1}=k_{2} = 5$, $d = 0.05\;\hbox {m}$. The parameters above are determined by both the mechanical limitations of the system and the environment.

The robot trajectory, the desired trajectory and the actual walking trajectories of the users are drawn in Fig. 11. According to the human-guidance algorithm discussed in Sect. 3.2, the desired path of the users shown in blue in Fig. 11 is the reference trajectory. In the reference trajectory, the central point in section C (in Fig. 6b) is selected as a reference point. The subjects need to adjust their directions step by step. The mean error distance of the steps along the entire path and over the twelve subjects is about 0.17 m with a standard deviation of 0.09 m. This result is better than that which has the mean error of 0.52 m with a standard deviation of 0.16 m in [12]. The average speed of the robot and the users are 0.18 and 0.21 m/s respectively. As presented in Fig. 12a, the 11th subject and the 12th subject have larger error distances than the other ten subjects. It conforms to the fact that they have not participated in the tactile perception experiment yet. Figure 12b shows the time evolution of error distance for the 3rd subject. From t $=$ 15 s to t $=$ 35 s, the user is turning a corner where the value of the error distance is relatively high. The error distance is generally larger when the user is walking on the curved path than the straight path because more motors may involve in the vibration display when the user is turning.

Fig. 11

The robot path, the desired and the actual paths of the users

Fig. 12

a The mean and standard deviation of error distance for the twelve subjects. b The time evolution of error distance for 3rd subject

Table 2 The error distance and the elapsed time of using the proposed system and the walking cane

In order to evaluate the performances of the designed guidance system, we also invite each participant to walk along the same L-shaped path once more. Traditional canes are widely used in the blind people. We compare the performances between the robot-guided method and the walking cane-guided method for relative evaluation. When a participant finishes the L-shaped path following test, he/she is invited to walk along the same path from the end to the start with a walking cane. The subject keeps blindfolded in the whole process. Note that the participant would be told vocally at the turn of the trajectory. The error distance and the elapsed time of using the proposed system and the walking cane are compared in the Table 2. The average distance error by using the walking cane is about 34 cm, which is almost twice the distance error by using the proposed system. The time cost by the subjects using the system is average 61 s, while the time cost using the cane is about 107 s. Thus the proposed system can truly guide the blind people more accurately and quickly.

To verify the adaptability of the system on the undulating terrain, the path following experiment is carried out using some 100 cm $\times$ 10 cm $\times$ 2 cm barriers shown in Fig. 13. The experimental results show that the guidance function is not affected even though the platform shakes slightly. It is because that the tracked robot has good adaptability of uneven ground and the desired location of the user is configured as a 20 cm $\times$ 20 cm square. The ability of passing barriers of the system depends on the obstacle avoidance capability of the tracked robot and the size of the barrier. If the robot can cross the barrier, the system can guide people normally as long as the user remains in the visual field of the Kinect.

Fig. 13

The experimental scene on the undulating terrain

Table 3 Questionnaire proposed at the end of experiments

Fig. 14

The mean and standard deviation of scores of the six statements in the questionnaire

4.3 User Acceptance Test

In order to evaluate the experiences of the users, a questionnaire using bipolar Likert-type five-point scales is filled out by each subject after the above experiments. As shown in Table 3, the questionnaire consists of six statements. The questionnaire, similar to the psychophysical tests in [24, 25], is designed to evaluate their comfort, opinion of feedback quality, perceived effectiveness of feedback, intrusiveness and flexibility of device, and overall performances. An answer of 5 means strongly agreement, whereas an answer of 1 means strongly disagreement.

In the questionnaire shown in Table 3, the first three questions investigate how much the users found the bracelet useful and the suggested cue to be easily recognized. The rest of the questions survey how much the users feel comfortable and safe, and the effectiveness of the proposed navigation system. The survey responses are shown in Fig. 14. The average scores of the first three statements are 4.00, 4.67 and 4.08, which respectively show that (1) the bracelet device is easy to use; (2) the bracelet device does not impede the users walking normally; (3) the haptic cues could be finely identified. As the results of the last three statements shown in Table 3, the user feels confident, safe and helped in the whole guidance process even if without any physical interaction between the robot and the human. In order to make comparisons, we change “the proposed system” in the last three statements in the questionnaire into “the walking cane”. The average scores of using the walking cane are respectively 2.92, 3.17 and 3.08. The average scores of using the proposed system are 3.75, 4.5 and 3.92. Compared with the walking cane, it can be concluded that the proposed system is user-friendly and less intrusive.

Fig. 15

The visual field when a box moves through the user and the robot

4.4 Anti-interference Ability Test

In the practical application, some intruders may interfere with the navigating process. It is necessary to warn the user of the existence of them. To verify the robustness of the proposed system, two cases are considered in the testing experiment: moving objects and walking people.

When a box passes through the user and the robot, the skeletal joints of the user may be obstructed by the box. As shown in Fig. 15 which is a critical case, the gray field is the visual field of the Kinect. Two equations can be given as follows:

$$\begin{aligned} \left\{ {\begin{array}{l} \frac{l_2 }{l_1 }=\frac{h_4 }{h_3 } \\ h_2 =l_2 \tan \beta -h_4 +h_5 \\ \end{array}} \right. \end{aligned}$$

(11)

where $l_{1}$ is the distance from the user to the robot, $h_{1}$ and $h_{3}$ are the heights of the whole body and the upper part of the body respectively, $l_{2 }$is the distance between the box and the robot, $h_{2}$ and $h_{5}$ are the heights of the box and the Kinect respectively, $\beta = (\theta + 21.5{^{\circ }})$.

Then an inequality about $h_{2}$ and $l_{2}$ is depicted by:

$$\begin{aligned} h_2 \le l_2 \tan \beta -\frac{l_2 h_3 }{l_1 }+h_5 \end{aligned}$$

(12)

Fig. 16

The process when a box moves through the user and the robot

According to the values of $h_{2 }$ and $l_{2}$, the obstruction problem can be divided into two situations. If the value of $h_{2}$ satisfies the condition in (12), the Kinect can still locate the user based on the present skeletal data even with some blocked joints. Otherwise, the Kinect cannot identify the user. In this case as shown in Fig. 16, the warning signal consisting of the vibrations of four motors in the bracelet will be sent to the user. And the robot will stop moving and wait until the Kinect re-identify the user to continue the navigation task.

Fig. 17

The process when a person intrudes in the same walking direction

Fig. 18

The process when a person intrudes across the user and the robot

As shown in Fig. 17, a person intrudes in the same walking direction. The system can guide the user correctly without any interference. The skeletal tracking of Kinect has a unique number (Tracking ID) for each tracked skeleton. A Tracking ID is assigned when a person is recognized in the interaction space. As long as the person remains in the interaction space, the ID will remain valid. The application can track the specified subject successfully. However, the intruder may walk across the user and the robot in a way shown in Fig. 18. The whole process can be divided into three parts. The first part is from the moment (Fig. 18a) when the intruder breaks into the visual field of the Kinect to the moment that the intruder starts to block the user. Owning to the unique ID of the user, the system can keep working. The second part is the period when the user is blocked by the intruder. The ID of the user becomes invalid, and the last skeleton data of the user is recorded in the application. The user will get the warning signal to stop walking and the system will stop working. The third part is from the moment (Fig. 18b) that the Kinect can identify the user again to the moment (Fig. 18c) that the intruder walks out of the visual field of the Kinect. The IDs of the two persons (the user and the intruder) both are valid. Comparing the skeleton data of the body with the recorded skeleton data in the second part, the closer one is the user. Based on these judging conditions, the application can identify the real user to guarantee that the system operates normally.

5 Conclusion

In this paper, a navigation service robot system for the visually impaired and people who need navigation in an unknown environment is proposed. Being different from the physical interaction typically used in existing navigation systems, a wearable vibro-tactile bracelet is designed as a perception interface. The haptic stimuli are considered for human–robot cooperative navigation, and a human-guidance control algorithm is presented. The subject is free to decide his or her own pace. By the vibration tactile feedback and the human-guidance algorithm, the user will perceive the vibrational warning information when deviating the ideal region. It can ensure that the robot guides the user to the destination safely and accurately. The tactile perception experiments evaluate the haptic bracelet. The experimental results show that the tactile indication method of the bracelet is effective. The trajectory following experiments on the flat and undulating ground are performed to verify the feasibility of the system. The results show an average error of 0.17 m with a standard deviation of 0.09 m along an L-shaped path on the flat ground and a certain degree of adaptability on the undulating terrain. The user acceptance tests are carried out in the experiments. The guidance system is proved to be easy to use. Also, the participants feel confident and safe in walking with the system. When some moving objects or walking people interfere with the guidance process, the system can cope with these cases and operate normally. This anti-interference ability verifies the robustness of the system.

Compared with other robot navigation schemes, the guidance system is user-friendly and less intrusive. The training time by using the proposed system is not long. The navigation system also has relatively small error distance, a certain degree of adaptability and robustness. In our future work, we will optimize the algorithm in human-guidance and expand the application scenarios in more complicated environment.