Keywords

1 Introduction

Blind people often have to learn to navigate through new routes, usually those that best fit daily needs. As they have limited perception of what is happening, they are advised to take known routes and to use points of reference (e.g. sounds, textures) to localize their selves. Commonly, blind persons firstly rely on others to learn routes, and then have to ask for help whenever they want to go through another way.

The assistive technology proposed in this paper intends to provide, for blind users, the information and assistance needed to safely navigate inside public spaces, therefore enabling them to navigate to unknown spaces/routes and providing an extra assurance when navigating through known spaces. Besides providing assistance during navigation, SmartNav also helps the user to get information about spaces, such as the services, products or promotions that may be of interest for he/she.

One of the main objectives of SmartNav is to equalize the access to information from everyone. As there is a strong amount of information accessed by visual means, there is also a strong amount of information inaccessible to blind people. The technology intends to replace the visual mean of receiving information with a voice channel robust enough to smartly interact and be trusted by blind users. Smart interaction is achieved by a keyword-based process that enables the blind user to promptly find the desired information.

The document is structured as follows: the next section reviews the related work on this field, the third section presents the SmartNav with more detail, the forth exposes the testing process and its results and the fifth concludes the paper.

2 Related Work

Some technologies were studied and developed concerning the issue of assisting the navigation of blind persons. The literature review enables to identify RFID and GPS as the technologies more often used.

BlindAid [5], is an RFID-based technology developed to assist blind people on navigation, by guiding them to desired destinations. Other works based in RFID technology are presented in [1, 3, 8]. This technologies use maps representations of the environment and the RFID sensors to locate the person. System [9] uses RFID technology for indoor navigation and GPS for outdoor. As known GPS has the problem of being extremely inaccurate for pedestrian navigation.

Regarding the issue of finding specific products/services, ShopTalk [6], BlindShopping [4] and RoboCart [2], are technologies developed to assist blind persons using blind-friendly mechanisms, such as the communication of information by voice or vibration.

In what concerns the level of assistance provided to blind people, the study presented in [7] concludes that the navigation skills acquired by blind persons should be included in the navigation system. Therefore, technologies intended to assist the blind during navigation does not require the utilization of complex sensors, as they are already aware of detecting structures, danger and moving objects or persons. Our work is based in this premise, SmartNav just instructs the user to navigate through the path, detecting objects and other structures rely on user abilities.

3 SmartNav

Taking into account features of the technologies revised in the literature (e.g. localization techniques, portable devices for blind persons), we defined the major features for the SmartNav. First of all, it must perform user positioning with enough accuracy to generate the correct instructions that can guide the user. The system must also comprise the information needed to help users concerning navigation and space, such as the services and products available. Additionally, the system intends to be a supplement of navigation techniques already used by users (for example, white cans and dogs). At the same time, the system must have low weight and size to be wearable by anyone.

Thus, we decided that the SmartNav must not force the user to carry another device, therefore it would be entirely developed to a smartphone (yet only developed for Android OS).

The SmartNav comprises four main functions: Interact, Inform, Guide and Position. Positioning is performed by placing beacons on the environment. Those beacons must be strategically placed in order to cover the entire space with radio signal. The technology can accurately position the user as long as there is signal surrounding he/she. The systems also uses GPS signal to guide the user in outdoor environments. Concerning indoor location, the type of beacons that can be used rely on the capacity of the user’s device to identify them. At this moment we developed the SmartNav based on Bluetooth beacons, because a wide range of smartphones already have incorporated the Bluetooth technology.

Interactions and Information play a special role in this system, as the target users are blind, there must be smart and efficient interaction methodologies. Similarly, in order to access and deliver the desired information, the information is treated in an intelligent and effective way. In the next two subsections we explained the interaction methodology and information treatment mechanisms.

3.1 Interaction

The mechanisms of Interaction play a special role in technologies designed from blind people. Once the SmartNav is intended to be used during navigation, it was developed without requiring any touch interaction (buttons). All changes of information are made by voice: both from the user and application.

Speech synthesis and speech recognition used are based on the Google API’s available for Android programming. Voice instructions (speech synthesis) are supported by the Android Text-To-Speech API, which enables to store or immediately play the speech. Since storing ASCII information is easily to be searched, SmartNav does not store information in voice format. Text-To-Speech API has several languages available (e.g. English, French, German, Italian and Spanish), not all languages or idioms are supported. The latest update on the API enables more a few languages, like Portuguese (Brazilian accent). The SmartNav was designed to accommodate new languages as soon as they are available.

The Text-To-Speech API uses a central queue to convert from text to voice instructions. That queue acts like a waiting list. There are two main methods of enter messages in the queue: place the text to convert on the last position of the queue and wait for its time, or forcing to convert and deliver it at the moment. The first method is used to deliver information about the environment surrounding the user, while the second method is used to deliver navigation instructions (e.g. “Turn left.”), which require a tight relation between generation of instructions and its deliver to user.

Speech recognition is performed by Google Speech Input API, which is factory installed on most Android devices. This API supports some more languages than the Text-To-Speech API, but once again, the SmartNav will be able to understand more languages as soon as the API is updated.

3.2 Information

SmartNav is able to inform the user of the available spaces and/or services surrounding he/she. This information contains the name of the points-of-interest available (e.g. spaces or services), a brief description of each one, and the promotions available at the moment (if he is navigating inside a shopping center).

Regarding the limitations of the target users, we designed a keyword-based approach to assist the user in finding the wanted information. The user is asked to introduce a keyword associated to the desired point-of-interest. This approach is used either to get information about the point-of-interest (service/place), either to select it as destination point and start assisted navigation (Fig. 1).

Fig. 1.
figure 1

Designed information handling approach.

Full size image

The above figure represents a diagram flow of the approach. Available points-of-interest are categorized into categories, subcategories and sub-subcategories. For example, the Fashion category can have the subcategories Clothing and Shoes, and subcategory Clothing can have Woman, Man and Children sub-subcategories. Every category, subcategory, sub-subcategory and point-of-interest is labeled with some keywords.

The introduced keyword is used to search for associated categories, subcategories, sub-subcategories or points-of-interest. Then a short list of associated categories (or subcategories, or sub-subcategories) or points-of-interest is delivered to user, referring first the points-of-interest. This list is labeled with numbers to easy selection of the user’s choice. For example, considering the keyword “Tennis”, the returned list would be “1 – Converse, 2 – Nike, 3 – Reebok, 4 - More choices”. The user is now able to choose one of the options, by saying its number. We opt by using numbers because they are easily understood by Recognizer part.

When a user chooses a point-of-interest, he/she can listening a short description of the point-of-interest, its latest promotions or defining it as destination point. If the uses chooses a category, the associated points-of-interest or subcategories are delivered the same way. To assist the understanding of information storage, Fig. 2 schematizes structures created and their relations. C1, C2, Cn are the categories available, SC1 to SCn the subcategories, SSC the sub-subcategories and the K’s are the recognized keywords. Each Category keeps its relation with associated subcategories and keywords. A similar relation is established for subcategories and sub-subcategories.

Fig. 2.
figure 2

Organization of information in SmartNav.

Full size image

Similarly, points-of-interest have some particular keywords that are added to the keywords’ structure. When the user introduces a keyword this structure is used to search for what is the interest of the user and what is available in the information system. At any time the user can introduce a new keyword and reset the informing process to the beginning. For example, after select a category and a subcategory, and received a list of the associated points-of-interest, the user can say a new keyword (instead of a number, used to select options from the list).

Since a destination point is defined, this informing cycle gives place to the assisted navigation.

4 Testing and Results

As referred before, the current version of SmartNav was developed for Android OS. Since it was designed to help blind people, it shows a simple visual interface with two big buttons: one at the top and other at the bottom of the screen (half size of the screen is used by each button). The button at the top is used to ask for help: either during navigation (to ask for navigation instructions to reach the defined destination) or just to request information about the space nearby. The button at the bottom is used to explore the information about the environment (this information is organized according to the specification detailed in Sect. 3.2). This can lead to the assisted navigation (as long as the user defines a destination point).

In order to evaluate the capabilities of SmartNav, a set of tests were performed. This tests intends to evaluate each function of the application: positioning, guiding, inform and interact. To test the positioning process, two scenarios were created. Fifty different positions were tested and in all of them, the position was delivered correctly. The guiding process was tested by defining 30 different routes and checking if the navigation instructions were correctly generated (giving the correct instruction in the right moment). Interaction was tested by extensively performing conversions from text to speech. Portuguese sentences and Portuguese sentences with loanwords were tested. The informing function was tested by navigating through the information process.

The testing phase showed the correct functioning of the positioning process, presenting maximum error of 20 cm from the correct position, and therefore, the guiding process is able to correctly generate navigation instructions in the right moment. Concerning to interaction tests, both Text-To-Speech and speech synthesis API used are robust enough to convert Portuguese words (form text to speech and versus), but some loanwords cannot be recognized or correctly converted to speech. Tests performed to the informing function showed its correct functioning, enabling the user to find any of the points-of-interest available.

5 Conclusion and Future Work

Blind people are deprived from a great amount of information because it is commonly delivered by visual marks. Usually locations, news and promotions are indicated with visual signs, so the blind person is incapacitated of being aware of any change made to his/her already known environment. They usually rely on others to receive this information.

SmartNav is a system specially designed for blind users, whose main objective is to assist on going and navigate inside public spaces. The system is supported by the user’s smartphone, thus enhancing its ability to use it (SmartNav is just another installable Android application). Interactions with the user are mostly performed by voice exchanges. In order to use the SmartNav, spaces must be provided with beacons, which allow determining user positioning based on radio signals.

The development of SmartNav’s prototype opened horizons to what it could be, as future work. We have assigned, as future work, the utilization of some sensors of smartphones, such as the gyroscope and accelerometer, and the creation of an online server with information about several public spaces. It will enable the user to be informed of available services and promotions without leaving the comfort of home.