Keywords

1 Introduction

According to the global prevalence estimates for blindness and vision impairment of the World Health Organization (WHO), visual impairment and blindness accounted for at least 2.2 billion people all over the world in 2019 and this number is continuing to increase as time passes. Approximately half of visually impaired people suffer from unaddressed refractive error, cataract, diabetic retinopathy, and other severe optical problem [1, 2]. Since one of the leading causes of blindness is cataract [3] which is easily found in developing countries, vision impairment is found more often in low- and middle-income regions than in high-income regions. Because of the vision loss, obstacles become a huge problem for blind people. Hence, external guidance such as humans, trained dogs, or supporting devices like white cane plays an important role in the decision making of blind people [4]. Although there are several psychological and social benefits for having a guided dog [5], training cost for a guided dog is relatively high. A previous study showed that majority of visually impaired people who participated in the study were willing to use personal navigation devices and would make more trip with such devices [6]. Due to those reasons, white cane, or blink stick is likely to be used by people with visual impairment [7, 8].

Many personal navigation devices for blind people, especially devices based on GPS, have been developed [9]. In addition, efforts have been made to create smart blind stick with ultrasonic sensor for obstacle detection for blind people [10]. Another smart blind stick, generally known as electronic travel aid (ETA), was reported in [11]. GuideCane [12], an innovation of Borenstein and Ulrich, has an array of the ultrasonic sensor at the end of a stick to detect obstacles. Another ETA was presented by Mahmud et al. [13]. This device used a PIC microcontroller (PIC16F90) to control an ultrasonic sensor, proximity sensor, and other components. In [14], Nada et al. reported a low cost and fast response smart stick.

Learning from the previous works, especially [11], in this work, we propose a Smart Blind Stick that can (1) detect and alarm obstacles as well as upward and downward stairs, (2) help blind people navigate better, (3) send via GSM temperature, humidity, speed, and location of blind user to a mobile app on smartphone of family members of the blind user, and (4) be low cost, low power consumption.

2 Materials and Methods

This proposed smart blind stick contains two main characteristics, which includes obstacle detection and location tracking. In terms of detecting hindrance, the ultrasonic sensor was used thanks to its wide range of detection based on ultrasound wave transmission and detection [15]. For keeping track on user, GSM module and temperature sensor provide the location and information of the user to another person, like a family member, by transfering these information to the family member's smartphone via SMS messages.

Figure 1 illustrates the working concept of our smart white cane. The structure includes three ultrasonic sensors, two of which are placed at the bottom of the cane for detecting obstacles in the nearest range of 2–10 cm. One ultrasonic sensor is mounted on the control box has a range from 30 to 150 cm (set to different ranges), which can detect things placed in the high position. A push-button (GSM switch) enables the user to send current status to a mobile application via SMS text messages. The control box contains a microcontroller in charge of controlling the system.

Fig. 1
figure 1

A sketch of the proposed system

Full size image

2.1 Stick Construction

According to a statistics of the Ministry of Health, Vietnam, the average height of Vietnamese people is 163.7 cm for men and 153 cm for women [16]. Thus, to make it easier to handle for both genders, height of this stick was designed to be approximately one meter. In term of material, a plastic rod made from polyvinyl chloride was used. This material is lighter and more durable than metal, and is able to withstand harsh weather conditions. The hollow rod allows reducing the weight and make room for wiring system.

At the end of the stick, a 3D plastic box was designed to fix the stick at an angle of 60° to the ground and protect internal electronic components. In the middle of the stick, there is a control box that was printed to cover a ultrasonic sensor, a GSM module, and a Arduino board. On the top of the white cane, there is a handle to make it possible for user to firmly hold the stick. In addition, there is also a push-button to send SMS messages containing location, coordinate, speed (m/s) of the person holding the stick.

Figure 2 illustrates technical drawing of our prototype. The first and second sketch shows the front and side view of the prototype respectively while the final one is an illustration of the whole system.

Fig. 2
figure 2

Technical drawing of smart white cane prototype

Full size image

2.2 Obstacle Detection and Alarming System

Ultrasonic Sensor

Ultrasonic sensors have many benefits including: high accuracy in range (2 cm–2 m), cost effective (about 1 $), good reliability in different environmental conditions. Compared to ultrasonic sensor, infrared one has some disadvantages, for example interference from other infrared sources (e.g., sunlight).

At the beginning, we used infrared sensor for detecting obstacles at close distances, because at close distance infrared sensor works well. However, when we tested the stick outdoor, infrared sensor alarmed whenever it was exposed to sunlight. Therefore, we chose ultrasonic sensors for all 3 positions (Table 1).

Table 1 A comparison of different obstacle detection sensors
Full size table

Ultrasonic sensor has working frequency at 40 kHz and requires 5 V and 15 mA current. Detection range is about 2–200 cm. Figure 3 illustrates working scheme of an ultrasonic sensor.

Fig. 3
figure 3

Timing diagram of ultrasonic sensor to measure the distance

Full size image

Ultrasonic sensors have 2 pins: Trigger and Echo. Specifically, the trigger pin initiates HIGH state in 10 µs, followed by a transmission sequence of 8 ultrasound burst at 40 kHz wave. Echo pin is activated immediately at the end of the eighth or the last burst, then acoustic wave bounces back from obstacle, generating a raise of voltage at the echo pin that can help to determine the time of wave traveled. Distance the wave traveled can be calculated from the relationship between distance, time, and velocity of ultrasound wave.

$$D = c{ } \times t$$
(1)

Accordingly, D denotes the distance from the detector to the obstacle, c denotes the velocity of sound at 20 Celsius, which is 343 m/s, and t denotes the time that wave traveled.

Converting unit of wave velocity to centimeter per micro second, we got 0.0343 (cm/µs). About time, since the wave traveled the path twice (one for transmission and one for reflection), time to travel from the sensor to the obstacle is half the total time. In summary, the formulation for calculating distance is:

$$D = t/2*0.00343\,\left( {{\text{cm}}} \right)$$
(2)

In reality, ultrasonic sensor performance is affected by factors such as shape/material of obstacles, angle between wave's direction and obstacle's surface.

Fig. 4
figure 4

Scenario in which echo wave does not return to the sensor

Full size image

For example in the case in Fig. 4, ultrasonic sensor cannot detect the obstacle although there is obstacle in front of the sensor.

For detecting obstacles in front of blind user, two ultrasonic sensors are used and cross-checked to increase reliability. There are various ways to alarm blind user when an obstacle is detected. In our design, we decided to alarm blind user using two senses of human: touch (vibration) and sound (buzzer) with different combinations (Table 2). The closer the obstacle the higher frequency of vibration/sound alarm. Besides, for detecting stairs going down, a third ultrasonic sensor is used. Electrical wiring of the obstacle detection and alarming system is shown in Fig. 5.

Table 2 Type of alarm signals corresponding to different distances of obstacle
Full size table
Fig. 5
figure 5

Wiring diagram of the obstacle detection and alarming system

Full size image

Arduino board has many versions with different performances and purposes such as Arduino Mega, Arduino LilyPad … Among them, Arduino Uno R3 is the most widely used one due to its low cost and flexibility.

The Arduino Uno R3 Board

This board is built with the ATmega328P microcontroller as its core using quartz whose oscillation cycle is 16 MHz. With this microcontroller, we have a total of 14 IO pins numbered from 0 to 13 (including 6 pin PWM, marked ~ before the pin code). In parallel, we have 6 pins to receive analog signals marked with A0–A5. These 6 pins can also be used as normal input/output pins (like pin 0–13). Of the pins mentioned, pin 13 is special because it connects directly to a status LED on the board.

The way to connect all components:

First, Arduino is powered by a 9 V-battery via 2 pins Vin and ground (GND).

  • 5 V pin (Arduino)—5 V pin (sensors).

  • GND pin (Arduino)—GND pin (sensors, buzzer, vibration motor).

  • 11 (Arduino)—V+ (buzzer).

  • 9 (Arduino)—V+ (vibration motor).

  • 2, 4, 6 (Arduino)—Trigger pin (sensors).

  • 3, 5, 7 (Arduino)—Echo pin (sensors).

For alarming, we want to alarm blind user not only by sound but also by vibration. Therefore, both buzzer and vibration motor were used so that blind user can recognize danger via two senses: touch and sound. Figure 6 presents the workflow of the whole system.

Fig. 6
figure 6

Flow chart of the obstacle detection system

Full size image

2.3 Data Transmission

For communication between the visually impaired people and their caretakers as well as location tracking, a GSM-based data transmission system was proposed in this work. When the blind user push the GSM button on the handle, temperature sensor (DHT11) and GPS module collect information about the speed of the user, environmental temperature, humidity, and location. Then, those data were coded in an SMS text message and send to the caretaker phone number. A specific Android-based application was used to read this code as well as display those data. The workflow of data transmission is summarized in Fig. 7.

Fig. 7
figure 7

Flow chart of data transmission system

Full size image

GSM Module

The Global System for Mobile Communications (GSM) is a popular protocol developed by the European Telecommunications Standards Institute (ETSI) to define protocols for second-generation wireless mobile networks (2G). Used globally by mobile devices such as mobile phones and tablets. When a GSM module is attached to a microcontroller (ATmega328), this allows the microcontroller to use the GSM module to communicate over a mobile network. To control the GSM Module, AT command was used.

SIM800L GSM/GPRS module is a miniature GSM module, which could be integrated into a wide range of IoT devices. You’ll be able to use this module to accomplish virtually anything a traditional mobile phone can: SMS text messages, create or receive phone calls, connecting to the web through GPRS. The module supports quad-band GSM/GPRS network, that means it works just about anywhere in the world. At the center of the module is a SIM800L GSM cellular chip from SimCom. The operational voltage of the chip is 3.4–4.4 V, which can be provided by a LiPo battery.

GPS NEO 6M Module

In reality, GPS receivers operate by finding out exactly how far they are from a range of satellites. At every given time, they are pre-programmed to learn where the GPS satellites are. The satellites relay information in the form of radio signals to the Earth about their location and the present time. Such signals mark the satellites and say where they are located to the receiver. The receiver then determines how far each satellite is by finding out how long it took for the signals to hit. It will identify your position on Earth after it has details about how far away at least three satellites are and where they are in the orbit. The process is called Trilateration [17] (Fig. 8).

Fig. 8
figure 8

The GPS trilateration technique, which applied signal received from at least three different satellites S1, S2, S3

Full size image

At the GPS module’s center is a U-blox NEO-6M GPS chip. The chip is less than the size of a postage stamp but can track up to 22 satellites on 50 channels and achieve the maximum degree of sensitivity in the industry by using just 45 mA supply current. Unlike other GPS modules, up to 5 position changes per second can be made with 2.5 m horizontal location accuracy. The U-blox 6 positioning engine also features a Time-To-First-Fix (TTFF) of less than 1 s. Power Saving Mode (PSM) is one of the key features that the chip has. It allows system power consumption to be minimized by swapping the ON and OFF receiver sections selectively. This significantly lowers the module’s power consumption to just 11 mA, making it ideal for applications that require high energy efficiency such as GPS wristwatch. The required NEO-6M GPS chip data pins are broken to a 0.1′′ pitch header. This includes pins needed for communication with microcontroller via UART. The module supports baud rate from 4800 to 230,400 bps with a default baud rate of 9600 bps.

Temperature Sensor

DHT11 is a common temperature and humidity sensor because it is affordable and easy to get data via 1-wire communication (digital 1-wire digital communication only). The sensor has a built-in signal preprocessor, making it convenient to use.

This sensor consists of a humidity sensor, an NTC temperature sensor (or thermistor), and an IC at the back of the sensor (Fig. 9).

Fig. 9
figure 9

Internal structure of humidity sensor

Full size image

The humidity sensor has two humectant electrodes to measure humidity between them. NTC temperature sensors or thermistor is used to measure temperature.

A thermistor is a resistor that changes its resistance to temperature change. These sensors are manufactured by sintering semiconductor with materials such as ceramics or polymers to provide greater resistance changes with only minor temperature changes. The term “NTC” means negative temperature coefficient, which means that sensor's resistance decreases with increasing temperature.

Android Application

Android application was developed using MIT App Inventor 2 [18, 19], which is a visual programming platform for designing applications for Android smartphones. This program was formerly developed by Google and is currently maintained by the Massachusetts Institute of Technology (MIT).

MIT App Inventor 2 allowed us to create an Android application via a web browser. By using its block programming language, we found it efficient to perform a project on this platform. After finishing the work, we can package our app and produce a stand-alone application to install on smartphones.

The app is defined to receive text messages only from the smart white cane; thus, other SMS messages will not affect the App. Regarding the function of the Application, since it got information about location, speed, temperature, and humidity from the GSM module on the white cane, we let the App show three variables: location, speed, and temperature. In terms of location tracking, there are two ways to show blind user's location. For people who prefer to use Google Maps app for searching location, we link the our Blind Stick App with Google Maps app so that app user can see blind user's location in Google Maps app. The other way is to have a map within the app by using the map function of MIT App Inventor 2 (BS Map) in which people can search for location together with other information.

2.4 Power Source

Our goal is to create a smart blind stick that can assist blind people continously for 4–6 h before battery recharging. To achieve this goal, we used a Lithium Polymer Battery (LiPo) in tandem with a charging circuit. Users can easily charge the battery through Micro-USB Type B charging port with relatively short charging time (in average 2–3 h). In our proposed system, there are two main parts which include (1) the data transmission system (Fig. 10) and (2) the obstacle detection and alarming system (Fig. 5). Each system is powered by a separate LiPo battery.

Power Supply for the Data Transmission System

At the beginning, we found it is challenging to power up the GSM module SIM800L. One of the most important parts of making the SIM800L module work is to give it enough power.

Depending on its status, the SIM800L is considered as a relatively current consuming device. Its current could be up to about 2 A during transmission burst. Normally, it is not required that amount of current but might ask for around 216 mA when calling or 80 mA during network transmission. The current consumption in different state is shown in the bellow datasheet summary (Table 3).

Table 3 The current consumption according to mode (according to the GSM SIM800L datasheet)
Full size table

An external power supply between 3.4 and 4.4 V (Ideal 4.1 V) is required because the SIM800L module does not come with onboard voltage regulator. External power supply should also be able to provide a 2 A source of surge current, otherwise, the module would shut down. Therefore, we decide to use a LiPo battery which is in the range of 3.7–4.2 V. This battery is perfect for the GSM SIM800L Module because it can provide the right voltage range even during 2 A spikes.

The other module is the GPS NEO-6M Module which has the ultra-low dropout 3V3 regulator (MIC5205) from MICREL and consumes only 45 mA supply current. Therefore, by using just one LiPo battery 3.7 V–3000 mAh, we can power the whole data transmission system. This design not only ensures sufficient power supply for the two modules but also creates convenience for users during long periods of use and easy to integrate into the stick compactly and aesthetically.

Power Supply for the Obstacle Detection and Alarming System

In the obstacle detection and alarming system, we used a sensor system (consisting of three ultrasonic sensors) to detect obstacles at different distances. In terms of alarm, we used two methods to alarm blind user including sound by using a buzzer and vibration by using a vibration motor. The obstacle detection and alarming system is powered by a second lipo battery, more specifically a LiPo Battery 3.7 V (2000 mAh).

The Battery Charging System

With the aim of developing a portable device, we used LiPo batteries to power the smart blind stick. To recharge these batteries, we used commercially available charging circuits (TP4056) that is highly efficient, durable, stable, and easy to replace. The TP4056 is a complete constant-current/constant-voltage linear charger for single-cell lithium-ion batteries. The charge voltage is fixed at 4.2 V, and the charge current can be programmed externally with a single resistor. The TP4056 automatically terminates the charge cycle when the charge current drops to 1/10th the programmed value after the final float voltage is reached. Other features include current monitor, under-voltage lockout, automatic recharge, and two status pins to indicate charge termination and the presence of an input voltage. In our system, we are using two battery chargers for the two LiPo batteries that power two different parts of the smart blind stick. With that, we measured battery run-time of around three-to-four hours at 60% operation and three hours at 90% operation. Battery charging time is around two hours for full battery charging. To charge batteries, we used a 5 VDC power adapter (5 VDC–2 A) which will charge the two LiPo batteries through two battery chargers TP4056 (Fig. 11).

Fig. 10
figure 10

Block diagram of the data transmission system

Full size image

The circuit of the Power Supply is represented in Fig. 11.

Fig. 11
figure 11

The battery charger wiring diagram

Full size image

3 Results and Discussions

3.1 Testing Procedure

One of the key components of our smart blind stick is the ultrasonic sensor (HC-SR04) which is used to detect obstacles. Based on its working principle, we designed experiments to test and calibrate distance measured by the sensor. After making measurements with the sensor, we measured and tested the ability to read the GPS value from the GPS-NEO 6M module, determined the error about the location value, accuracy, and stability of the system. All experiments were performed indoor in conjunction with measurements in outdoor environment, where the stick would be primarily used.

3.2 Obstacle Detection Experiment

Results of the Proposed Method

First, we experimented to determine the accuracy of the HC-SR04 sensor’s ability to measure distances. For reference, a steel ruler was used to measure the actual distance in centimeters. Room walls were used as obstacle from which the sensor will measure distance. The sensor was moved away from the wall and then actual distances was measured using the steel ruler. Distances measured by the sensor will be shown on serial monitor screen of Arduino IDE. We designed experiments based on experiments that [20] performed. Totally, we took 14 sample distances within the desired working range. For each distance, we measured three times to compute the mean value and standard deviation, as shown in Table 4. From the results, we analyzed percentage of error (ε) for each distance and average percentage of error (εa) as shown in Table 4. Percentage of error was computed by using the Eq. (3).

$$\varepsilon = \frac{{D_{m} - D_{s} }}{{D_{m} }} \times 100\%$$
(3)
Table 4 Data from the experiment that tests the accuracy of the HC-SR04 sensor
Full size table

Table 4 shows a very good agreement between steel ruler and HC-SR04 sensor. In all measurements, error percentage is less than 5% and so is the average. Based on Table 4, we can see that the further the distance between the object and the sensor, the greater the percentage of errors. Moreover, the percentage of errors can still be affected by external environmental conditions (i.e. Temperature, Humidity, Materials, etc.). Therefore, we proposed the measurement distance for our system should be in the range of (0–100 cm) in order to provide the most accurate results as well as minimize the percentage of errors. That will help to improve user experience as well as ensure user safety.

We also plotted the relation between fixed distance (Dm) measured by ruler and distance measured by HC-SR04 (Ds). From the plot, we obtained a regression equation where regression coefficient was 0.9996 using Microsoft Excel software, as shown in Fig. 12.

Fig. 12
figure 12

Plot of fixed distances measured by ruler versus measured distances measured by the HC-SR04 sensor

Full size image

The value of coefficient of determination (R2) is 0.9996 as extracted from Microsoft Excel software, and the slope of the regression line (i.e. regression coefficient, a0) is 0.9488, which signifies that the sensor was performing very well.

In addition, we measured voltage of the ECHO pin of the ultrasonic sensor (HC-SR04) at different distances between the sensor and the obstacle. As shown in Table 5 and Fig. 13, we see that the larger the distance, the higher the measured voltage.

Table 5 Voltage measurement at different distances
Full size table
Fig. 13
figure 13

Distance versus measured voltage

Full size image

Next, we tested our smart blind stick's ability to detect different obstacles made of different materials (10 times for each one). The results are shown in Table 6.

Table 6 Ability to detect different obstacles made of different materials
Full size table

Results in Table 6 reveals that the ultrasonic sensors are good for various types of material as long as the surface is flat. Whereas rough surface is difficult for sensor to detect (as explained Fig. 4). The sensor could detect only five times in the total of ten times tested. In general, the stick works well in most of the cases.

Comparison with previous works

The range of obstacle detection in our proposed method has shown promising results compared to the previous method. This can be analyzed from the system of Mahmud et al. [21] which also contains three ultrasonic sensors for detecting obstacles and uses vibration to inform the users. The threshold of our method is higher than the one from Mahmud et al. (20–100 cm and 30–70 cm respectively). Moreover, although there are three levels of vibration to help the user notice the different in distance, the previous method can lead to misapprehension, which has been improved in our prototype by using both buzzer and vibration. Nada et al. [14] represented an effective way to create a smart white cane by using infrared sensors and voice record for alert. This method provide the cane to have the priority in detecting obstacles in less than 20 cm while our method can identify at the minimum distance of 20 cm; however, Nada et al. show limited ability to work with object that is more than 80 cm from the stick. The comparison is illustrated in Table 7.

Table 7 A comparison with other devices
Full size table

3.3 Data Transmission Experiment

For our proposed system, we also experimented to validate the accuracy of our GPS module (GPS NEO 6M). In this experiment, we took 100 location measurements both in good conditions and difficult conditions. Table 8 shows standard deviation of latitude, longitude, altitude, and the localization error in meter.

Table 8 Standard deviation of latitude, longitude, altitude, and the localization error of GPS NEO 6M in both good conditions and difficult conditions
Full size table

The localization error was computed by Eq. (4):

$$d = R\sqrt {a^{2} + b}$$
(4)

where R is the radius of the Earth (6371 km) and \(a = \Delta \lambda \cos \varphi_{m}\); \(b = \Delta \varphi\). Accordingly, \(\varphi\) denotes latitude, \(\lambda\) is longitude and \(\varphi_m\) denotes the mean value of latitude.

From the table, we can observe that the precision of GPS data depends on both the GPS receiver and the observing conditions. The radio interference also might have a very pronounced impact on the accuracy of the results. Another key thing is that the GPS module is unable to work under indoor environment because it needs to communicate with at least 2–3 satellites to receive the signal. So it is impossible to get GPS signal under indoor environment. This problem prevented the stick from transmitting location data to smartphone app when user moves inside buildings and future work is needed to address this issue.

4 Conclusion

This paper present a design and implementation of a smart blind stick that can detect and alarm the presence of obstacles for visual impaired people. Furthermore, the stick is able to send information including location of the blind user and temperature and humidity of the surrounding environment to a remote smartphone app so that family member of the blind user can track the blind user's location when needed. Multiple tests were conducted, and the results showed high accuracy and high reliability of the device.

Our design offers a low-cost and robust system for navigation and location tracking for blind people. In future work, weight of the stick could be reduced for better user experience.