Keywords

1 Introduction

One of the major problems that blind people encounter concerns their mobility. While public transportation is getting more accessible, the destination might still be challenging to navigate without guidance. Before going to unknown buildings, it is important to familiarize yourself with the environment. This is especially true for people with blindness, as they can only perceive acoustically their immediate surroundings when entering a building. In large public buildings, information about the location of departments and offices is often provided by visual maps placed in the hall or in front of the building. These maps are intended for sighted visitors and not accessible to blind people. At this point, blind visitors would need help or preparation in advance to get knowledge about the layout of the building and where to find what they are looking for. When they lack the needed information, blind people often tend to limit their mobility and avoid areas they are unfamiliar with [1]. Tactile maps are one way to bridge the information gap for this target group. It has been shown that blind people are capable of building a mental map and orienting themselves like anyone else, if they have the opportunity to get an accessible source of information and maps [2]. However, the information which can be included in a tactile map is very limited as tactile information, such as text in Braille or tactile symbols, need more space than text and symbols in visual maps. Therefore, systems that allow interaction with maps, e.g. by touch, are a good way to provide additional information.

In this work, we address the inaccessibility of indoor maps to blind people. To achieve this, new specific features have been added to the existing audio-tactile system TPad [6]. The system allows to put a tactile indoor map on an iPad Pro fixed by a frame and retrieves additional audio information about the map via touch to get a fast overview of the building. The system is shown in Fig. 1. Our new system provides auditory icons of the most important information and voice commands to enhance the interactivity of the system.

Fig. 1.
The image depicts a frame, tactile indoor map and a T Pad.

The TPad system composed by an iPad Pro with the TPad app, a tactile indoor map and a frame to fix the map on the tablet.

Full size image

2 Related Work

A lot of research has already been conducted to provide accessible indoor maps to visually impaired people. Calle-Jimenez et al. [3] proposed a method to produce on-line SVG-based indoor maps that are designed to be explored by keyboard and screen readers. This approach has the disadvantage of not providing spatial information as the user loses the two-dimensional view of the map. The estimation of distances becomes difficult and even the relative positions of rooms can become challenging.

A more promising approach would be to use touch-sensitive devices coupled to audio feedback to provide spatial information. The GraViewer [4] used audio feedback to present floor plans on an iPad. The users can explore the plan with one finger and receive speech and audio feedback to navigate the map. This approach allows to explore a plan. However, the information about an indoor map must be reconstructed exclusively from many punctual audio feedback. This might not be enough for a good understanding of the explored material. Another study [9], also using image sonification, despite having good results, showed that participants preferred tactile graphics over sonification. This preference leads to the conclusion that a combination of both sonification and tactile graphics seems to be a good solution, since the use of tactile documents alone is not interactive.

A preliminary study [7] describes how adding vibration to complement audio feedback can improve the accessiblity of digital maps. Their approach allows zooming in a map, which is an advantage because more information can be stored. A disadvantage is that understanding the map with the help of vibration seems to be a relatively demanding task. The task was easier when the map was zoomed in, suggesting that one of the disadvantages of vibration feedback is the low resolution achieved with this technique. Another disadvantage is that most tablets do not include vibration motors, limiting the range of devices that can be used. In a similar manner, the GraVVITAS (Graphics Viewer using Vibration Interactive Touch and Speech) project [5] used a combination of audio and vibration feedback. Devices without vibration motors need to be used in combination with a haptic ring. The last two approaches share the disadvantage of not allowing two-hand exploration of the map, as the vibration can be felt by only one finger on the display.

The solution proposed in this paper addresses the issues discussed above and in particular meets the following requirements: two-hand exploration of the tactile map, a high resolution of the tactile component and an excellent degree of interactivity with the system.

3 Creating New Features for TPad

The first version of the TPad audio-tactile system was developed and tested specifically for the use with educational material [6]. In this section, we shortly describe the original TPad system. Then, we show how we improved the interaction methods by introducing the following new features to TPad: (i) two levels of information, and (ii) voice commands, i.e. map overview, unvisited elements, audio guided search. We also report on how we optimized the use of TPad by making it compatible with the VoiceOver screen reader.

Description of the Basic TPad System. TPad consists of a mobile application, installed on an iPad on which the digital document to be consulted is loaded. A frame allows to easily fix the tactile document on the tablet and thus can be explored with two hands like a normal tactile document. With a double-tap on a graphic element, it is possible to hear the text associated with that element, if the text is saved as description tag in the SVG input file.

Two Levels of Information. Each element present in the SVG file has two attributes: element title and description. The title was used to report to the user a short information about an element after a first double-tap, a second double-tap on the same element returns more detailed information contained in the description attribute.

Auditory Icons. In indoor maps, many elements are standard features such as stairs, restroom or entrance. We integrated the option to choose “Auditory icons" instead of natural language. Exploring the map with a single finger, when an interactive element is touched, an auditory icon will be automatically played. This allows the user to explore the map more quickly.

Voice Commands. To enhance the interactivity of the system, the following three features have been added which can be triggered via voice commands:

1) Map overview command. This command lists all interactive elements contained in a map. It allows the user to get an overview of the map by hearing a summary of all available elements they can interact with. The elements are grouped by type to reduce verbosity of the speech output.

2) Unvisited elements command. When exploring an interactive tactile map, it is possible that the users misses some information because they did not find some elements. The unvisited command returns a list of elements not yet explored. The app gives also a short description of the first three items of the list. At this point, the user can choose to be guided to one of these unvisited elements using the following search command.

3) Audio guided element search. At any time, the user can enter this mode using the voice command “search", for example “search entrance". In this mode, all interactive elements of the map are deactivated, except for the target one, to not disturb the search. By sliding a finger on the map first horizontally and then vertically (or vice versa), the user will receive two different sounds that will allow to locate the element on the map. The frequency of the sound changes depending on the distance from the target. Please note that this modality can be employed to signal temporary elements (e.g., construction work, temporary hazards) that do not have a tactile symbol and are only included in the updated digital version of a map.

Compatibility with VoiceOver. The current version of TPad was written in Swift using SwiftUI. One of the main benefits of this framework is the support for native accessibility features of the iPad. TPad is now compatible with VoiceOver and doesn’t need an own accessibility feature. In this way, we avoid changing settings which is challenging for persons with visual impairment [8].

4 Indoor Map Interaction Using TPad

In this section, we describe in more detail how a user interacts with the system. We will assume the scenario in which a specific room is searched, as well as a way to reach it from the entrance. After loading the map and placing the corresponding tactile graphic on the device, a first possible interaction would be the “map overview" voice command. Then, using the “audio guided element search" feature, the main entrance to the building can be located quickly and with certainty. Now, the user begins to explore the tactile map by alternating between two-handed and single-finger exploration modalities. With the second modality, it is possible to distinguish the various interactive elements thanks to the auditory icons, and with a double tap more information will be read on demand. Once the user have located the room, the entrance, and the route from one to the other, they can choose to fully explore the map by invoking the command “unvisited elements" to get the list of the unexplored map elements.

5 User Study

To evaluate the described improvements on the TPad system, imaginary indoor maps from three buildings of increasing complexity were created. The complexity is defined by two parameters: the amount of rooms and the amount of symbols. The first map had 7 rooms and 4 tactile symbols, the second map 7 rooms and 9 symbols, and the third map 15 rooms and 13 symbols. The third map is shown in Fig. 2.

Fig. 2.
The image depicts two indoor maps. The left one is the digital version that has colors and the right one is the tactile version.

The third indoor map used for the study. On the left the digital version of the map (SVG file). On the right the tactile version for the embosser. Circles with a dot represent restrooms, open triangles entrances, sequences of four rectangles stairs and of two rectangles steps, open squares elevators, empty circles receptions, filled circles POI.

Full size image

5.1 Participants

Four blind people participated to the study. The age of the participants was in the range of 23–28 years old. All the participants were familiar with screen readers in general and with VoiceOver, the screen reader installed on the TPad system. One of them reported to use tactile materials weekly, two monthly and one even less. Only one participant reported to use tactile displays monthly, while the others reported to use them only seldom.

5.2 Method

Before starting with the tasks, participants were introduced to the system and its features. Using an indoor map of a simple building, participants familiarized with the system. When participants felt ready, they were asked to perform the same tasks for each of the three different indoor maps with increasing difficulty. We provided the participants with a precise workflow to make sure that all available features were used.

Task 1. First they were asked to use the command “map overview" to get a summary of the map content, then they were free to explore the map using the audio-tactile system, without time limit.

Task 2. The participants were asked to describe the given map, while being allowed to continue using it. The time to provide the description was limited (60 s for the first map, 90 s for the second map, and 120 s for the third map). The audio of the descriptions has been recorded.

Task 3. The next step of the study was to ask the participants to search a given specific element in the map with the search command and describe the way out of the building from that point.

Task 4. For the first two maps, a search for a non-tactile elements was requested, to see if the implementation of the search was sufficient for non-tactile elements.

In the end, the participants were asked to use the command “unvisited elements", to note how many elements have not been explored. Before concluding the study, participants were asked to respond to evaluation questionnaires (SUS, NASA-TLX, features ranking, open questions).

6 Results

The descriptions of the three buildings provided by the participants (task 2) were transcribed and a score was calculated. Each map element contributed at most four points to the total score. One point was attributed for mentioning it and one for mentioning its correct location. Then, one point was given if the user recognized a particular shape of the room. The bottom left room in the building in Fig. 2 is an example: the room has a narrow entrance and gets bigger after that. The last point was attributed for mentioning additional information that is contained in the audio descriptions. If something is described wrongly a point is subtracted. This score is only intended to reflect the amount of correct information contained in the descriptions, and not weighted by its importance.

We also compared the maps’ descriptions collected during the user study with the descriptions of the same buildings made by four sighted people. For that, we created visual maps with the same information. The comparison with the sighted participants’ descriptions does not show significant differences in quality, as shown in Table 1. However, it must be mentioned that the blind participants were allowed to explore the maps as long as they wanted to compensate the lack of sight, as it is common in such studies or situations.

Table 1. Average score for the blind and sighted participants for each map.
Full size table

The evaluation of the different tasks is very promising, as the descriptions given by the blind participants were of high quality and accuracy. Sometimes parts of the building were missing because the participants had forgotten them, but mostly this was due to the time pressure of the task and they were able to mention the missing elements afterwards. Also the description of the way out of the building after finding a specific location on the map (task 3) did not cause any difficulties. All participants were able to describe the correct way out. We report here one of the transcriptions relative to the third map (Fig. 2) starting from the restroom in the middle of the map: “I walk out of the restroom to the top (of the map), then I can either (1) go left, very briefly along the corridor until it turns down, continue down the corridor, down the steps to the lobby, through there, down and to the entrance. The other option (2) is to walk from the restroom to the right along the corridor until I reach the side exit.".

In a System Usability Scale, the system received a score of 83.75 (range 72.5–95). The results of NASA-TLX investigation, point out low values (lower than 10/20) for the sub-scales effort, frustration, mental, physical and temporal demand. For the self-estimated performance the average value was 16/20. Both the tools of investigation seem to indicate a usable system with an acceptable demanding load on the users.

The participants have been asked to rate the new features of the system in a scale from one (worst) to five (best). The average scores are reported in Table 2. Most of the features received a high ranking. Only the listing of all objects in the map seems not to be considered an interesting feature by the participants.

Table 2. Average rating the participants gave for the different features.
Full size table

There is a limit on how much information should be placed on a map, otherwise some users may get lost in the complexity. However, the third indoor map, which was the most complex one, was considered as the upper limit of complexity by one user, one user was undecided, and the other two were confident that the system may be used with more complex buildings’ maps. We observed that the system was quite well accepted by the participants of the study. They felt it was useful and some even enjoyed it. The main criticism to the system was about the lack of an alternative to the voice commands to interact with the new features of the system.

7 Conclusions and Future Work

We created an audio-tactile system that is intended to help visually impaired people in exploring indoor maps to prepare a visit to unknown buildings. A study was conducted testing the usability of the system and analyzing the quality of the participants’ buildings descriptions. All blind participants were able to accurately describe most of the information contained in the maps, and the system performed well in terms of usability and workload. The descriptions were compared with a sample of descriptions from sighted users, and no obvious differences were found.

In future work, it would be interesting to investigate how participants manage to orient themselves in unfamiliar buildings after using the TPad to plan a visit. In addition, the possibility of splitting complex buildings into two maps could also be explored, as the participants’ opinion differed on whether this would help or hinder the continuity of the map.