1 Introduction

The Internet was initially created with the purpose of the interconnection of research laboratories. Eventually, as the years passed, became what we know today: an integrated global network used by billions of people. Moreover, the Internet is in constant evolution because it is an open communication system based, capable of combining several areas of knowledge and enables the creation of new types of services and protocols.

Today, the Internet is used in different fields being the Internet of Things (IoT) one of the more relevant. This term was used by the first time in 1999, by Kevin Ashton, during a presentation where he proposed the usage of the RFID technology in the P&G supply chain. He suggested the use of the Internet and addressing technologies to improve the flow of products, with faster and more reliable movements, without the direct interference of the human being. Ashton stated that computers, and therefore the Internet, are dependent on individuals to create data. However, the human being is limited to retrieve and treat data of the physical world [3].

With IoT technologies, the physical objects of everyday life have the capacity of interacting with each other and with the physical environment in which they are inserted. Such objects must contain embedded systems, software components, sensors, and connectivity to exchange information among these objects across the network. In [4] the main concepts, technologies and standards of IoT are pictured. As shown in Fig. 1, the IoT meets the “Things”, “Internet” and the “Semantic” oriented visions.

Fig. 1
figure 1

Internet of Things paradigm by [29]

Full size image

In order to support the wide range of IoT services and applications, various protocols, models and architectures are emerging and being studied to meet such needs. Smart cities, smart energy, smart spaces, urban mobility, home automation are just a few of the many potential scenarios in which this technology will be present. With this in mind, it’s easy to visualize the benefits of these applications may bring to society [4].

Urban mobility in smart cities is a growing area that deals with technologies being conceived into the IoT paradigm. The Assistive technology (AT) field could benefit in this scenario. AT is a term used to define the full range of equipment, services, strategies and practices designed and applied to minimize the problems encountered by individuals with disabilities [11]. The proliferation of AT and its impact on the lives of people with disabilities is upward [33].

According to [33], the advent of the IoT in the fields of mobility, healthcare, and education has challenged the IoT applications into current AT services and solutions. According to the authors, solutions are still in a primitive stage and clear guidelines and benefits are have not been developed yet to ensure the quality of the services provided to people with disabilities. Motivated by these elements, an integrated framework with an IoT architecture customized for an electronic cane (a cane is a device used by the visually impaired) is presented in this paper. Moreover, this research aims to assure more autonomy and independence for people with visual impairment in a Smart City.

In Fig. 2, the four main areas of interest of the research are depicted. These areas will be explained in the next section for a better understanding of the proposal presented in this manuscript.

Fig. 2
figure 2

Areas of interest in the proposal

Full size image

The paper is structured as follows. In Section II, the literature background and the related works are discussed. Section III describes the proposed IoT architecture for the electronic cane in a Smart City context. Section IV describes the development stages. The operation of this framework is validated in Section V, showing the route trials and the analysis of the feedback from three blind users. Finally, Section VI presents the conclusions of this work and the future lines to be addressed.

2 Literature background

The vision of the term Internet has become a challenging task for researchers to clearly identify the optimal potential of Internet usage [28]. IoT makes computing truly ubiquitous, a concept initially put forward by Mark Weiser in the early 1990s [36].

Moreover, improvements in electronics are enabling widespread deployment of connected sensors and actuators (the IoT) with predictions of billions of smart objects by 2020. Indeed, IoT is rapidly evolving, and there is a need to understand challenges in obtaining horizontal and vertical application balance and the key fundamentals required to attain the expected 50 billion connected devices in 2020 [31]. Popularity of IoT application is due to the dominance of internet users, the development of smartphone technology and mobile communication standards [27, 31].

2.1 Internet of Things, architectures and smart cities

The IoT is a novel concept that envisions a near future, in which the objects of everyday life will be equipped with sensors, microcontrollers, transceivers for digital communication, and suitable protocol stacks [4].

Due to the growing developments in digital technologies, Smart Cities have been equipped with different electronic devices based on IoT. IoT shall be able to incorporate transparently and seamlessly a large number of diverse and heterogeneous end systems while providing open access to selected subsets of data for the development of a plethora of digital services [48].

In the last few years, a significant research effort and technological development have been devoted to the IoT domain targeting smart city concepts [14]. The primary one is the exponential growth of devices/smart objects that can participate in an IoT infrastructure [44]. The second most notable reason is population growth and urbanization trend. According to the United Nations, the urban populations will grow by an estimated 2.3 billion over the next 40 years, while as much as 70% of the world's population will live in cities by 2050 [9]. Several challenges arise due to density of population (pollution, inequality, poor health, climate change, decarbonization). Therefore, cities need to be adequate to human needs, to survive as platforms that enable economic, social, and environmental well-being [26].

New IoT applications are enabling Smart City initiatives worldwide. They provide the ability to remotely monitor, manage and control devices, and to create new insights and actionable information from massive streams of real-time data [5]. The essential components of urban development for a smart city should include smart technology, smart industry, smart services, smart management and smart life [30]. IoT heralds a vision of the future Internet where connecting physical things through a network will let them take an active part on the Internet, exchanging information about themselves and their surroundings [6].

In this context, smartness of a city is driven and enabled technologically by the emergent IoT [4]. Smart City strategies must take the stakeholders' opinion into account and seek a compromise between their views and the implementation of the strategy. It can therefore be assumed that narrowing the gap between the stakeholders' vision of Smart City initiatives and the implementation of certain projects may make a decisive difference to the success of these strategies [17].

Different Smart City architectures based on IoT have been proposed for providing new services and applications to the society [18]. These different kinds of architectures enable a broad portfolio of smart city services and applications, while their effectiveness depends on the architectural solutions to pass from data to services for city users and operators, exploiting data analytics [5, 10, 26]. Moreover, several devices, together with IoT can be used to define a framework in this context. Today smartphones include powerful sensors, which can be accessed through customizable applications. In [10] the authors present an architecture for using sensory data generated by smartphones to improve the public transportation systems. In [26] the proposed framework encompasses the complete urban information system, from the sensory level and networking support structure through to data management and Cloud-based integration of several systems and services. This framework forms a transformational part of the existing cyber-physical system.

When planning a Smart City, mobility is a crucial issue to develop more sustainable cities [7, 12]. The term sustainable is related to the city itself, as an effort for improving the efficiency of urban services from different perspectives: environment, energy, transport [47]. In [1] the authors focus on how to develop their smart city concept, especially smart mobility, considering two different cities: Taipei in Taiwan and Surabaya in Indonesia.

However, the term Smart City is not only related to connectivity, mobility and technology, but it should consider how the citizen’s life is. Consequently, a Smart City should provide high accessibility features for everyone to support an effective way to move around the city, considering different citizens’ skills. In this sense, many researchers have investigated various problems that people with disabilities face in the urban space considering some improvements that assistive technology should have to assure, like autonomy and independence [39, 40]. Related with these trends, new concepts as Human Smart Cities or Inclusive Smart Cities are emerging. The Human Smart City concept proposes to develop a citizen-driven, smart, all-inclusive and sustainable environment, with a new governance framework in which citizens and government engage in listening and talking to each other. An Inclusive Smart City reinforces the participation of everyone recognizing the diversity of citizens, struggle against the segregation of minorities, and try, as much as possible, to eliminate, not only physical but also digital, barriers.

Finally, considering the fundamental principles of Inclusive Smart Cities proposed in [40], the research looks for a framework for making the urban space decipherable to people with visual impairments and including AT tools in the urban space attempting to find opportunities to maximize their usability.

2.2 Electronic canes

White canes are broadly used by visually impaired individuals, particularly in urban environments. Despite its many advantages, this device does not detect obstacles at head and chest height. Such barriers are commonly found in urban environments due to the lack of accessibility issues, and they can lead to accidents and injury [35]. For this reason, the development of walking assistants for visually impaired people has become a prominent research field [33]. Electronic Travel Aids (ETA) collect information from the environment and transfers it using sensors, cameras, and smartphones [25].

Sensor-based walking assistants commonly provide surrounding information through audio signal or vibrations. Those frameworks depend on the gathered information to recognize an obstacle and avoiding it by calculating the distance between users and obstacles. Ultrasonic and light sensors are widely used in these sorts of solutions. Examples of electronic canes available for final users are: the Ultracane, the Bat-K Sonar, the Tom Pouce and Télétact, the Laser, the iSONIC and the Smartcane [13, 16, 23, 25].

System architectures of sensor-based walking assistants for visually impaired people are not usually integrated into smart spaces. While numerous frameworks have been developed, they are still limited in their scopes and most efforts have been devoted to avoid obstacles [23]. Prototypes include an electronic cane with features enhanced by contextual information from the GPS (global positioning system) and surrounding information obtained via haptic feedback [24, 40] and an electronic cane with ultrasonic sensors and camera and audio output [2]. In addition to mobility devices for obstacle avoidance, high tech orientation and navigation aids have been developed [41].

Elmannai and Elleithy [19] present a comparative survey of the wearable and portable assistive devices for visually-impaired people. Authors discuss the progress in assistive technology developments for this group of people and present guidelines to ensure a satisfactory performance. Guidelines include performance, reliability, simplicity, wireless connectivity and economic accessibility.

Kim and Cho [29] present a quantitative evaluation of a smart cane prototype. The evaluation was focused on users with visual impairments, and in-depth interviews were conducted. The results showed that the smart cane approach was more effective in avoiding obstacles than a white cane, but there were also several potential usability problems. Based on the results, the authors devise some guidelines for smart cane prototypes considering user satisfaction.

2.3 The electronic cane project

The electronic cane project was developed at the Universidade do Vale de Itajai (UNIVALI) to help visually impaired people in terms of orientation and mobility in urban spaces. This is a pioneer project in Brazil [43], granted (grant number MU86010425) by the National Institute of Intellectual Property (INPI), in 2018. The electronic cane was inspired by haptic technology [13] and differs from the traditional long cane, basically, by having an electronic system incorporated into the handle, which emits haptic signals (sounds and tactile) when locating a barrier (obstacle) above the waistline [42]. The basic functioning of the electronic cane can be seen in Fig. 3. In this figure, the user is warned about an aerial obstacle (public phone) in its path, deviating it to avoid a collision.

Fig. 3
figure 3

The electronic cane detecting an obstacle

Full size image

The project began in 2005, when it was selected in the “MCT/Finep – Assistive Technologies” contest. The project is continuously improved, supported by governmental agencies. Nowadays, around 50 users test the device in everyday activities.

In [23], the authors demonstrated the validity of its proposal for detecting aerial obstacles. However, this device was not designed to extract relevant information from the environment. So, it was impossible to embed new applications, for example, to inform the proximity to specific places and remote monitoring.

The development of walking assistants for visually impaired people has become a prominent research field. Electronic Travel Aids (ETA) collect information from the environment and transfers it using sensors, cameras, and smartphones [25]. Those frameworks depend on the gathered information to recognize an obstacle and avoiding it by calculating the distance between the users and obstacles. Ultrasonic sensors are widely used in these types of solutions. A wide range of portable and wearable electronic travel aids have been developed to enable visually impaired mobility in urban spaces. However, only a few of them have gone beyond the prototype stage, and the long cane is still the main mobility aid [23].

In addition, while numerous frameworks have been developed, they are still limited in their scopes [25]. In fact, there have been quite a few attempts to harness the available contemporary technological advances in mobility assistive devices for blind users [42].

Two solutions were used as a base of knowledge and inspiration in this work:

  • I-Cane: The I-Cane [24] is an electronic cane created by the company of the same name. The cane adopts technologies such as proximity sensors to detect obstacles above the waistline and features GPS (Global Positioning System) for the localization and navigation of the visually impaired.

  • Handisco: Handisco [22] is a company that develops electronic devices for the traditional white cane. Such devices feature GPS, audible warning devices, proximity sensors and BLE (Bluetooth Low Energy) wireless communication module. Government agencies help this company to install around five thousand IoT devices in strategic locations of the city of Nancy, France.

A suitable architecture for the IoT paradigm requires the implementation of several technologies in computing, communications, and data mining [34]. In [33], an overview of the IoT and AT integrated applications is provided, along with recent trends and issues relevant to accessing technology for people with disabilities. According to the author, five building blocks of the IoT architecture should be considered: Radio Frequency Identification (RFID), sensors, sensor network, cloud computing, and application. Recent IoT devices for people with disabilities consider at least three of these building blocks. In [15] the author suggests three levels of IoT architecture: perception, network, and application.

In this work, an IoT architecture for the electronic cane (Bengala Branca Eletronica) was proposed. Such architecture was based on the proposals of [4, 6, 49]. The implementation of this architecture allowed the electronic cane to get three new main features. The first one is the ability to connect to environment devices, called sensor nodes, receiving the coordinates of their fixed location. The second one is alerting the visually impaired by sounds (beeps) when it is close to any one of these sensor nodes. The third feature enables it to send the node sensors coordinates to a web application for remote monitoring.

3 Proposed framework and architecture

This section presents the technical aspects of the proposed framework and the architecture implemented for supporting this framework.

There are two main architectures available for building solutions bases in IoT: Reference Architecture (IIRA) and Internet of Things-Architecture (loT-A). IIRA was proposed by a group of companies including AT&T, Cisco, General Electric and IBM and was created mainly to serve as a base reference for IoT solutions in the industry. In the other hand, IoT-A was made by an European consortium of companies formed in 2010 with the same purpose, but instead of creating an industry based architecture, they created a reference aimed to any IoT solution, offering a detailed view of all aspects needed to implement a solution. In terms of popularity and adoption the IoT-A has been described in detail and extended. Since its 2012 launch, it has been synchronized with the Io'T community and incorporates multiple views. In contrast, the IIRA still aims for feedback and further detailing. Anyway, both architecture proposals must be known for the creation of a reliable and maintainable structure. In resume these architectures can be analyzed regarding their capabilities and layers according to three perspectives: Semantic, Internet or Things oriented [4, 48] (see Fig. 1 for more details).

Furthermore, depending on the scope of the project or the problem domain being addressed, architectures were focused on different aspects or in sub-domain of IoT. Therefore, due to a considerable heterogeneity of application domains and consequently the requirements, the approaches to the architecture specification differed between the projects thus resulting in different architectures, comprised of several components and protocols. This resulted in limited interoperability between the systems which combined with often different terminology used also made discussions between the domains challenging [21, 32]. In summary, identifying and structuring an architecture or model is a long, tedious process with much negotiation to abstract from specific needs and technologies. Such a reference can serve as an overall, generic guideline; not all domain applications will require each detail for real-life implementation [45].

Taking these issues into account and considering that more and more devices are getting connected to the Internet day by day, the authors have implemented an internet perspective based on IoT-A architecture, like [4, 49]. In this context, the next logical step is to use Internet and its associated technologies as a platform for IoT devices’ communication (i.e., sensor and actuator networks, embedded devices, electronic appliances and digitally enhanced everyday objects) [20].

Figure 4 illustrates the different interactions between the components of the framework. Firstly, the IoT devices and the electronic cane provide mobility support by means of the obstacle detection, sound and tactile warnings (embedded in the cane). Secondly, the electronic cane and the mobile device's connection provides advanced alerts for users and mapping support. Finally, mobile devices allow the connectivity with the central server for monitoring and positioning the IoT devices.

Fig. 4
figure 4

Interactions of the components of the proposal

Full size image

The proposed framework is based also on the concept of Web of Things (WoT) [8]. WoT is intended as a unifying application layer for the IoT, linking together multiple underlying IoT protocols using existing web technologies. The key point is that this doesn't involve the reinvention of the means of communication because existing standards are used as an infrastructure for smart objects [36].

WoT seeks to counter the fragmentation of the IoT, making it much easier to create applications without the need to master the huge variety of IoT technologies and standards Therefore, WoT is a computing concept for creating a decentralized IoT where everyday objects are fully integrated with the Web [8, 46].

Moreover, the framework is also based on the one proposed in [15], but including one more level: the haptic level. Therefore, the framework proposed is composed of four levels: an application level for connecting applications with the lower levels, and internet level for ensuring connectivity, the perception level for obstacle detection and finally, the haptic level for acoustic and tactile warnings. The different levels of the framework and their relationships are shown in Fig. 5.

Fig. 5
figure 5

Global framework of the proposal (adapted from [15])

Full size image

As it can be seen in Fig. 6, the key building blocks of the framework have been organized in five layers, following the ideas of the IoT-A project, for the interoperability of the heterogeneous components as one coherent network. The middleware and application layer have been developed using Amazon Elastic Compute Cloud (so using a common provider for both layers). The communication or internet layer is based on Wi-Fi and 3G/4G technologies (so using communication standards). The device layer is two-fold, edge layer for sensors and gateway layer for electronic cane device. For the security of data communication, the framework includes an https connection, and also the communication between sensors nodes and the cane could be encrypted (but latency issues would need to be considered).

Fig. 6
figure 6

Low-level IoT architecture proposed for the electronic cane

Full size image

Once the framework has been defined, an integrated architecture has been developed following the layers of such framework. The low-level architecture for the electronic cane ecosystem is shown in Fig. 6. It consisted of five layers, which are edge technology, gateway, Internet, middleware and application. In this section, these five layers and their implementation are discussed in detail.

3.1 Edge technology layer

The sensor nodes are located in the edge technology layer, which are microcontroller hardware based and embed a radiofrequency module, see Fig. 7. The purpose of these devices is to transmit the geographic coordinates of their fixed location through a radiofrequency communication link with the electronic cane.

Fig. 7
figure 7

The sensor node architecture

Full size image

Microcontroller The microcontroller is responsible for storing the coordinates of the fixed location of the sensor node device, also controlling the radiofrequency module to communicate with the electronic cane. An Arduino Pro Mini was used in the sensor node.

Radio Frequency Module The purpose of this module is to provide the sensor node with the ability to communicate with the electronic cane (gateway). It was used the nRF24L01 module, which is an ultra-low-power transceiver operating at 2.4 GHz.

3.2 Gateway layer

The gateway layer has three functionalities. The first one is the communication with the sensor nodes to receive their geographic coordinates (fixed locations). The second one is the connection to the cloud system through the Internet, which arises through positive connectivity, identity and security checks, thus allowing sending and receiving data to the web application. The third functionality is responsible for alerting the visually impaired, with audible signals, if it is close to a sensor node. The electronic cane hardware was increased by a Wi-Fi module—a radio-frequency module (nRF24L01+) the same as used by the sensor node—to allow communication between the two parts. In addition, a buzzer emits the sound signals.

The Wi-Fi module is responsible for controlling the communication with the sensor nodes. It verifies the origin of the geographic coordinates received and then send these coordinates to the web application. The NodeMCU module, which is an SoC (System on a Chip) ESP8266, was added to the electronic cane's onboard system, which enabled it to both the radio frequency module and the buzzer.

3.3 Internet layer

In the Internet layer, there are the technologies that provide the connectivity of the Cane with the Internet. In this work the Wi-Fi module embedded in the NodeMCU provides a connection with a smartphone, in a router mode, featuring 3G/4G mobile data technology.

3.4 Middleware layer

The middleware layer is presented in the cloud and is organized by an MQTT (Message Queuing Telemetry Transport), the broker and an MQTT client connected to the database. The main functionalities are the abstraction of the data shared between the electronic cane and the web application, the data storage and the access control of the clients. In this project, the t2.micro instance of Amazon Web Services were configured with an Ubuntu Server 18.04 LTS to compose the cloud system. The MQTT broker is the Mosquitto version 1.4.15, due to its open-source feature and to the possibility of enabling a series of security settings, such as user password and Access Control List (ACL). Finally, to create the MQTT client it was used the MQTT.js, written in JavaScript.

3.5 Application layer

The application layer is formed by the back end, front end, and the database of a web application. A series of libraries and tools were used in the construction of the back end:

  1. 1.

    The Node.js is a run-time environment that allows the use of JavaScript on the server-side.

  2. 2.

    The Express is a framework used to code server programs.

  3. 3.

    The Object Document Mapper (ODM) Mongoose is used to facilitate access to the MongoDB database.

MongoDB is the database chosen for this project to accomplish higher scalability and speed to the system. Several tools were also used for the front-end construction, among them: The Handlebars, a template engine that separate HTML files from the content written in JavaScript; the Google Maps JavaScript API, which allows to the creation of mapping applications using functionalities like map creation, markers, routes, among others.

Finally, the MQTT.js library allows the creation of the MQTT Websocket client. In this proposal, this client was implemented at the web application to receive the Cane coordinates in real-time, and thus, showing the current position of the user in the map. Besides that, the client was also implemented to allow the inverse communication by giving the user's application the possibility to send MQTT data back to reproduce specific sound signals at the electronic cane.

4 Development

This section presents the conceptual aspects of the proposed framework. The literature background section has shown the importance of making urban spaces decipherable to people with visual impairments. Assistive technology develpments attempts to maximize the user experience [40]. For this reason, a new framework based on an IoT architecture customized for an electronic cane has been implemented. The goal of this framework is to provide more autonomy and independence of people with visual impairment in a Smart City scenario.

The following functional requirements were considered for the proposed system:

  • FR01: The electronic cane should alert the user is the cane is close to a sensor node.

  • FR02: The system should allow sending data through the web interface.

  • FR02: The system shall allow the visualization of the information enclosed in the sensor nodes through a web interface.

  • FR03: The system should allow remote verification of the user's location through a web interface.

  • FR04: The system shall allow the visualization of the places visited by the user, through a web interface.

  • FR05: The system shall estimate the path between two consecutive places.

As well as the nonfunctional requirements:

  • NR01: The system should receive data from the electronic cane via WiFi.

  • NR02: The system should provide the connection to the cloud through the Internet,

  • NR03: The system should provide a positive connectivity, identity and security checks.

Different libraries were used to encode the sensor node firmware: the RF24, which allows the implementation of all the functionalities of the radiofrequency module; and the SPI library, featuring the communication with the Arduino Pro Mini. Initially the program activates the radiofrequency module transmitting its fixed geographic coordinates. If the cane is located at a minimum radial distance of 3 m from the sensor node, it accepts the geographic coordinates and sends back an acknowledgement to the sensor node, which receives it and waits for 15 s to resume the sending. This time was set to avoid the duplicity of messages, considering that the individual would take a few seconds to approach to the object, and moving away from it.

Figure 8 depicts the behavior of the sensor nodes firmware.

Fig. 8
figure 8

Behavior of the sensor nodes firmware

Full size image

The following libraries were used at the firmware level of the electronic cane: ESP8266WiFi to provides the Wi-Fi connection; PubSubClient to features the MQTT protocol; and the RF24 and SPI libraries previously explained. Initially, the Wi-Fi module attempts to connect to the smartphone's preconfigured network and then securely connects to the Mosquitto broker. If the connections follow without errors, the program initiates the radiofrequency module in receiving mode allowing the electronic cane to collect the coordinates sent by the sensor nodes. When a coordinate is received, the ESP8266WiFi realize two procedures. The first one is to register the sensor node coordinates in a specific topic into the broker, following the patterns of the MQTT protocol. Once the registration is done, the coordinate data becomes visible to the subscribers of the topic, in this case, the web application. The second action is the activation of the buzzer in order to alert the user by a sound signal, indicating the proximity to a sensor node.

As stated earlier, the cane is also capable of communicating in the opposite direction, which means receiving the messages sent from the web application. This mechanism works by the MQTT subscription instruction, executed by the firmware, causing the electronic cane to receive messages from the web application (see Fig. 9).

Fig. 9
figure 9

Interaction between the cane and the web application

Full size image

The web application was developed considering four functionalities:

  1. 1.

    Authentication, by the user registration and login.

  2. 2.

    Real-time data representation (geographic coordinates sent by the electronic cane) in a 2D map.

  3. 3.

    Graphical representation of the estimated path performed by the user.

  4. 4.

    Recording the history of places visited by the visually impaired.

5 Experiment

The electronic cane detect obstacles at chest and head-level based on echo detection. It gives a haptic feedback to the user, warning about potential collisions. It also preserves the functionalities of a traditional white cane. In this proposal, the device is now integrated into the surrounding environment through IoT, comprising smartphone connectivity and support.

The electronic cane has been evaluated at all stages of development. The first device was evaluated at the Santa Catarina Association for the Blind Integration (ACIC), in 2009 [41]. A second model was evaluated at the Brazilian Association for Assistance to Visually Impaired People (LARAMARA), in 2011 [17].

Both studies were descriptive. Participants performed different tasks, previously defined, regardless of time consumed and troubles found during tasks execution. Semi-structured interviews were conducted with all the users to register their experiences using the electronic cane.

Recently, under the contest 84/2013 “MCTI-SECIS/CNPq—Assistive Technology”, new devices were built, and a novel usability study was performed. It was based in a two-part methodology involving an experimental investigation (quantitative) of the electronic cane performance (in real locations with real obstacles) and a questionnaire (qualitative) to explore user satisfaction [23].

The results showed that the electronic cane can successfully inform users about the presence of obstacles at height. Users had a reasonable degree of satisfaction with the cane properties such as its weight, ease of folding and sound. Participants asked for sensor improvements, considering the possibility of collecting more information from the environment. In addition, they stated that any modification would not significantly increase the cost of the device [23].

The extent of user satisfaction and performance of the current prototype and the expressed interest in purchasing the Cane, justifies a continuous developing of the electronic cane project. The results obtained in this study are compatible with the feedback previously obtained using the former version of the prototype.

The architecture described in this paper, based on IoT, was tested in a real environment, in the city of Itajai, Brazil. Three male blind users, aged 29, 38 and 61 years old contributed to the preliminary evaluation of the proposed architecture. Two users were born blind (29 and 38 years old), and all the participants were trained in orientation and mobility programs. Moreover, the participants in the experimentation have been collaborating for a long time in the electronic cane project. Therefore, they have tested former versions of the Cane and have previous experience with electronic devices such as the one included in the proposal. A researcher and an expert in white canes and in mobility and orientation training supervised the experiment. We know that the number of users is not enough in order to provide meaningful conclusions, but these are the only users of the electronic cane in the city of Itajai (where the study was conducted). We are planning to assemble new devices in order to include more users in this experiment. Ethical approval was obtained from the University of Vale de Itajai in Santa Catarina, Brazil.

In this stage, the study was descriptive and qualitative assessments were made in order to explore how participants perform the walk. An exploratory visit to selected places around the university was made and a route was designed for the participants. Then the guided tour, under controlled conditions, was performed with each user. Only three sensor nodes were used in this work. They were attached in beacons. The route was performed in 10 min and the interviews took 15 min. Figure 10 shows one of the users of the electronic cane.

Fig. 10
figure 10

Experiment with a user of the electronic cane. Credit by Daniel Queiroz

Full size image

5.1 Communication between the cane and the sensor nodes

Each sensor node was configured to report the geographic coordinates of its fixed location. When the user approaches, the sensor node first recognizes it and alerts by sound signals (beeps) emitted in the Cane, then enabling the transmission of its geographic coordinates. All the attempts approaching to the sensor nodes were successful.

Figure 11 shows the communication messages between a sensor node and the electronic cane using the terminal of the Arduino IDE. Figure 11a shows the attempts of the sensor node sending the coordinates to the Cane. In the two first lines fail occurred because the Cane is out of range. Please, notice that this is not really a fail. In the third attempt the sensor node detects the Cane and sends its coordinate to the embedded electronic. Figure 11b depicts the successful reception of the sensor node coordinates by the Cane. Then the sensor node location is sent to the Mosquitto broker. It means that when the user approaches to the distance of three meters, the transmission succeeded. This distance was configured to provide enough time to react when user approaches to a sensor node when was walking. This test was performed in the Robotics Laboratory at Univali using one sensor node and the Cane.

Fig. 11
figure 11

a The node sensor is waiting for the proximity of the Cane; b the electronic cane received the geographic coordinates from the sensor node and sent it to the Mosquitto broker.

Full size image

Latency between node sensors and the electronic cane were analyzed in 100 trials. The latency values depend on the radiofrequency range (2.4–2.525 GHz) and external factors, such as signal interferences and physical barriers. The latency was measured, considering a reach of 3 m, resulting in a range between 40 and 160 ms.

The web application was developed following the Model View Controller (MVC) standard allowing the inclusion of new functionalities in a simplified way, with high scalability, both at the level of requisitions and at the storage level.

Figure 12 shows the route used in the experiment. The web application brings this information after performing the login and displays on a map the probable route travelled by the user. This estimation is made by considering the last two geographic coordinates stored in the database. It is important to notice that the more sensors, more accurate and complete this information will be. The inclusion of more IoT sensors will be addressed in future works.

Fig. 12
figure 12

Web application interface

Full size image

After the tests, a semi-structured interview took place with each participant. Users were asked in an open communication between the interviewer and the interviewee. They were conducted to explore more open-ended questions, stimulating a discussion with the interviewee blended the straightforward question and answer format [37]. Participants were asked about their experiences using the Cane with this new functionality. The structured part of the interview was the evaluation of the responses by getting the positive and negative feedback and analyzing them.

As positive aspects, the participants highlighted:

  1. 1.

    Haptic signals did not overlap with other senses when nodes proximities were detected.

  2. 2.

    The connectivity added to the cane contributes to getting more spatial information, improving locomotion.

On the other hand, suggestions were made by the participants in order to improve the design:

  1. 1.

    It was suggested the use of speech messages, instead of sounds, to advice the proximity of the node location.

  2. 2.

    The participants worried about whether the cost of the cane would increase because of the IoT based architecture.

It was explained that hardware modifications increase the cost of the cane by only 3.50 USD. Moreover, each node sensor costs 1.50 USD. It is expected that a sensor network will be provided by smart cities infrastructures. Those sensor networks may be availed by other applications and would have also other functionalities, like sensing temperature, chemical gases, etc. In that scenario, the electronic cane will take advantage of this infrastructure, having a new functionality to improve locomotion of visually impaired individuals.

Therefore, there are initial capital and operational costs. However, when the project was fully implemented, it will reduce costs and resource consumption for the city and also will more effectively and actively engage with their citizens [38]. Moreover, it is important to remark that this infrastructure could serve to other uses, services or needs, and through a cloud service, it is feasible to obtain the system scalability. Future approaches can be focused on obtaining any government agreement for large-scale implementation.

6 Conclusions

This research presents a framework conceived to enhance the interaction with the urban space, including assistive technology resources for finding opportunities to improve the locomotion. The research aims for an integrated framework with an IoT architecture customized for an electronic cane, assuring more autonomy and independence of people with visual impairment in a smart city scenario. When designing the framework, the authors have considered the fundamentals of Inclusive Smart Cities, proposed in [40].

The innovation of this work consists in the implementation of an IoT based architecture, embedded into an existing electronic travel aid developed in Brazil: the electronic cane project. This study discussed the inclusion of three new features in this device. The first feature is the ability to connect the cane to devices available in the surrounding environment, that we called sensor nodes. Second, the electronic cane receives the geographic coordinates of different fixed locations and alerts by sounds (beeps) when it is close to any of these sensor nodes. In addition, it was included the capability of sending the node sensors coordinates to a web application for remote monitoring and other uses. The electronic cane is also capable of communicating in the opposite direction, which means receiving the messages originated from the web application. The article approaches two related works: the I-Can and the Handisco IoT solution.

The IoT architecture has enabled electronic cane to broaden its capabilities, allowing it to collect data from IoT devices and send them to a web application. It enables the ability of recording and informing the user of the cane about the places visited, also drawing the estimated pathway. The web application performs high scalability, both at the level of requisitions and the storage level. The architecture achieved a satisfactory result by receiving data from the nodes and sending messages to the broker, even with the mediation of a smartphone and the Internet connection. The MQTT protocol guarantees the exchange of information with the Mosquitto broker. Moreover, when the minimum distance of three meters is reached, the messages are transmitted correctly, without significant latency.

The architecture was tested in a real environment. Three blind users of the electronic cane participated and an expert in orientation and mobility techniques supervised the experiment. Observations and interviews were conducted to evaluate this proposal and qualitative assessments were registered. Results show the contribution of the electronic cane project as a travel aid. The study also points out that the modification made to the original prototype provides better interaction with the surrounding environment contributing to mobility and orientation. Besides, the communication between the cane and the sensor nodes promises new functionalities beyond the ones conceived in this work.

Future works will be directed to improve the IoT architecture, in particular, the number of sensors nodes will be increased and speech messages, instead of beeps, will be sent back to the user to identify the actual location of the node. Also, sensor nodes locations in a path will be stored in the microcontroller embedded in the cane. This feature could be useful in case of the lack of a stable Internet connection. The web application should also be improved to include new features. The electronic cane project is also relevant as the first attempt to enhance the functionalities of white canes carried out in Brazil. In addition, future works will consider the inclusion of more users to perform the usability evaluation of this architecture.