D-Braille: A Digital Learning Application for People with Low Vision

Abstract

Visually impaired people have trouble getting visual information, affecting their learning progress and day-to-day life. The Braille system was introduced to teach blind and low vision people to read and write. However, teaching them how to use and understand Braille is also essential. Low visioned people usually use Braille, but due to lack of resources and more time consumption, it becomes difficult for them to learn Braille, affecting their learning growth. Therefore, this paper presents a mobile application interface called “D-Braille” for people with low vision. This application has been designed specifically for people new to Braille who want to learn Braille from the beginning. This application helps them to learn and feel simultaneously. They will gain access to tutorials to verify the spelling of terms and practice letters on the Braille display. The application’s main features are learning to read and practicing reading letters and words. Such a user-friendly application will enable users with low vision to learn to read letters and words with the braille dots on their fingertips connected by the Braille keyboard.

1 Introduction

Globally there are 2.2 billion people who have near or distant vision impairment. Of those 2.2 billion, half of the people are not even aware of vision impairment [1]. Different kinds of visual impairment that vary between individuals can impact several developmental zones, notably personality, intellect, language, and cognitive development. In addition, the social development of children is influenced negatively as they cannot understand the nonverbal cues and cannot make eye contact [2].

Low visioned people usually use Braille, but due to lack of resources and more time consumption, it becomes difficult for them to learn Braille, which affects their learning growth. Also, the resources available can become costly for some people as their costs can vary from 500–2000 dollars. The Braille system was introduced in 1824 for low vision and blind people and was adopted from the military-based “Night Reading” system. It is a system that consists of touch reading and writing for blind and low vision people. Braille consists of 6 dots, two horizontal and three vertical dots. It’s read from left to right, and both hands are used in the process. It also contains punctuation equivalents and provides symbols to indicate the grouping of characters. The reading, on average, is 125 words per minute and can go up to 200 words per minute in some cases. Therefore, all the printed words can be read and enjoyed by the blind and visually impaired people just like everyone else [4]. Over the years, many other methods have been introduced to support blind or visually impaired reading. Several of them, though, were elevated copies of printed letters.

From a basic typewriter created in the nineteenth century to assist blind people in writing intelligible letters to a mobile phone app that lets blind people “see” and comprehend their environment, technology has advanced tremendously over the years. By increasing freedom and safety, assistive technology can improve the quality of life for visually impaired people. Additionally, these technologies can alleviate their anxiety of social isolation by encouraging them to take a trip outside their usual location and interact socially [5]. Braille reading and writing are the two essential needs for visually impaired or blind people. But, as we see, it is difficult for a child or an adult who just started learning the Braille system to read and write simultaneously. Therefore, they need to be trained to do both jobs together.

Touch-based applications are in trend as everyone uses a smartphone in their day-to-day life. Both android and IOS phones come with voice-over technology developed to help blind and visually impaired people to access the phone’s main features and give them access to use smartphones just like other ordinary people. Braille touch mobile and mBraille are the applications developed for smartphones and provide the user with the ability to learn Braille using their mobile phones. mBraille allows the user to learn to write letters or characters, practice writing the letters in the application, and play some learning games. It was developed in Bangladesh and allowed the users to learn to write English and Bangla letters [6].

Such applications have been implemented for many years, but most of the applications only train writing Braille letters and words. And hence, there is a need to create a generic platform where visually impaired people can learn how to read Braille letters and words. A user-friendly application that will enable the users to learn reading letters and words with the feel of the braille dots on their fingertips.

Therefore, in this paper, we present the first, to our knowledge, a mobile application interface called “D-Braille” for people with low vision. This application has been designed specifically to teach read and write letters, numbers, and words and practice them to learn better. The main objective of this application is to provide a user-friendly interface where the user will be able to use the application without facing any problem interacting and understanding how it works, and learn how to use braille in the easiest way possible. We have conducted two assessments to measure the task solving abilities of low vision people carefully. Our results show that our was more engaging to interact with and promoted the low vision people more feature to use compared to existing applications.

2 Related Work

In recent years, the concept of developing solutions to teach Braille has gained popularity. VinithaIt et al. [3] developed a software solution for teaching braille letter recognition to young blind people. It lets users engage with the computer by interacting with the integrated NFC-tag blocks with Braille characters fetched on both sides. Young people interact with the system through a tangible interface to provide information and receive audio feedback through a voice-based interface. The virtual interface consists of a block with Braille characters embossed on the sides, and an NFC (Near Field Communication) tag is attached to the bottom. The blocks are small enough to be operated with small hands and sturdy materials such as wood and plastic to allow children to survive rough handling. The groove-based surface enables multiple blocks to be placed simultaneously and hides the NFC tag reader linked to the PC. When the young child puts the block on the surface, the Software detects the letters and provides proper auditory feedback.

Mobile braille touchscreen [2] was to help users, significantly visually impaired students, to learn Braille. The braille touch mobile app has voice narration to enable users, especially the visually impaired, to use the app. The narrator’s voice uses Google’s text-to-speech technology. The braille touchscreen application on a mobile phone has six main functions: learning, practice, writing, translation, configuration, help. There is also an additional feature called Speed Writing Compute. The virtual braille keyboard layout in the Braille Touch Mobile app has six buttons and a 3$\,\times \,$2 binary matrix layout (Fig. 1).

Fig. 1.

Virtual braille keyboard [2]

mBraille application mobile app was designed for VIS-focused elementary education as shown in Fig. 2. Besides auditory and vibration responses for elementary school students or trainees, this app includes all characters and words in Bengali and English Braille. Student users will gain access to tutorials to verify the spelling of terms and practice letters in Braille. The application is operable on Android phones. The screen contains six dots, which the user can press. Some features of the application are [7]. It includes letters to learn, which offer users to learn letters. This sub-function will explain which points must be pressed to write a particular letter. Practice letter provides the necessary tools to practice the letters learned in the “Learn Letters” sub-function. However, this application works only on low-cost Android phones. However, these Android phones might not be available in the future, and therefore, more alternatives need to be explored.

Fig. 2.

mBraille user interface [7]

From the research mentioned above, it is clear that many educational applications have been introduced in recent years. However, no generic educational mobile application exists for people with visual impairment to learn and practice letters, words, and sentences at a low cost. Therefore, we first design and implement a generic mobile application to learn and practice braille in this paper. Also, we evaluate this application with eight low-sighted people to check the usability.

3 Research Methodology

To develop an effective design for the digital braille application, we first interviewed two teachers at the University of Education in Germany. They are teaching Braille and the five low-vision people from a non-government organization. Through this interview, basic needs were identified. The students were using the Braille display to learn Braille. But, because of COVID-19, they could not go to the schools and learn Braille in person with teachers. They also added that these Braille displays were expensive and not affordable because of the high cost. The needs were collected, and initial designs were created and evaluated.

3.1 Software Architecture

The system is made up of several components. The primary application has three user-facing features. The first feature, “learn and practice reading Letters,” enables users to read and practice letters while receiving audio feedback. The second feature, named “learn and practice reading words,” allows users to learn and practice reading letters. The third and most essential lesson is “Learn and practice reading numbers,” which enables users to learn and practice reading numbers. The application uses the database to store and retrieve data for the users. The system employs a Braille keyboard to provide input and receive output. The application uses device Bluetooth and voice-over features to connect to the Braille keyboard. It is required to enable the voice-over function in the IOS device to use the app since this allows the app to provide audio feedback. The application’s initial feature teaches the user to read letters from “A” to “Z” one by one. Upon completion of each letter, it announces which letter it was. The microservice architecture of the application is shown in Fig. 3. Furthermore, users can go back and go to the main menu.

Fig. 3.

Microservice architecture

Application Flow. The main features of the application are designed in English and German language. When users launch the application, they have six alternatives to choose from. On the screen, there are six buttons:

• 1. Learn Reading Letters

• 2. Practice Reading Letters

• 3. Learn Reading Words

• 4. Practice Reading Words

• 5. Learn Reading Numbers

• 6. Practice Reading Numbers

Since the application is for visually impaired people while navigating through the options, the voice feedback will help the users know which button they are on; an animation will indicate where they are navigating. To navigate, they have to use the Braille keyboard. Once they know which mode they want to learn, they must press okay from the Braille keyboard.

Learn Reading Letters, Words and Numbers

• In these mode, the user navigates through the numbers 1 to 6 using mobile application, while the application’s audio feedback tells them which digits are dotted as well as what is clicked on the screen. Every letter or dot touched will be felt on the braille.

• To check which letter is dotted, the user presses the “NEXT” button.

• In the new screen, voice feedback pops out, displaying which letter, word or number it was.

• The user now has three alternatives to select from:

  • Either they can go back and recheck the numbers if they are confused by clicking the “PREVIOUS” button.

  • Or they can go to learn the following letter by pressing the button “NEXT”.

  • Or they can go to the main menu to change their learning mode by pressing the “HOME ICON” button.

    The same process is done when they choose to learn reading words or learn to read numbers.

Application Design and Implementation. The application is implemented with react native framework and Microsoft Azure SQL database. The application’s home page design, seen in Fig. 4, is the first screen visually impaired users encounter when they open the app. The home screen has been designed to make it more noticeable to them. The colors blue and yellow were chosen for the design because they are more visible to visually impaired individuals. The animation on the middle of the screenplays by indicating which button the user has clicked, with the assistance of voice feedback. On the main screen, users have four options to choose from and two others on the next page: learn Reading letters, practice reading letters, learn reading words, practice reading words, and learn reading numbers practice reading numbers.

Fig. 4.

Home screen of the application (Color figure online)

Figure 5 displays a standard screen which is the first screen of all the three learning modes of the application. It consists of seven buttons, of which the first six are numbers from 1 to 6. Some of them are colored in blue to show that the button is clicked. For example, it is dot 1 and the button “NEXT” to see the result on the second screen. For example, in here the result is “Dot 1 is shown and represents letter a”. On the other hand, voice feedback will always mention what is on the screen and what is clicked in order to guide the user. Then the user can click “NEXT” for the following letter or “PREVIOUS” to repeat the letter, Or click the “HOME ICON” to go to the main screen.

Fig. 5.

Letter ‘a’ dots (Color figure online)

Figure 6 displays a screen which is the second part of the application, and it is learning words. It consists of learning short-form words by starting with the words beginning with the letter A. Firstly simple words like “About,” “Above,” “After” will be shown to the users. Then, according to what he will select, “About” in this scenario, he will feel dot 1 and 2 on the braille representing the short form of “About.” Then that explanation will be given at the final screen when he clicks “NEXT.”

Fig. 6.

Learn word starting with A

Fig. 7.

Learning number

Figure 7 displays the screen of the third part of the application, and it is learning numbers. It consists of learning numbers where first an explanation of how number works is given. It is different from other learning concepts here since it is a combination. Users must first know to read the letter before learning numbers.

Fig. 8.

Learn reading number one

Figure 8 shows the result user has clicked dot 1, which is letter a, and is asked to add a number sign which he was thought in Fig. 7. Then after adding the number sign on the second screen, the third screen shows the result and explains how the process was done.

4 Usability Testing and Verification

• Participants: The application is evaluated with the five low-vision people interviewed earlier and two teachers who are expert users of Braille display. Here, we conducted a study to understand the application from the user’s perspective.

• Apparatus: The usability testing was performed in the presence with the D-Braille application running on iPhone.

• Procedure: At first, we introduced the application and explained the concepts in detail. After that, we gave them approximately 15 min to get comfortable with the application. Some of the participants already started to give the feedback as follows:

  • The error messages should also come via a voice and haptic feedback

  • Color of the button can be darker

  • Size of the animations and button can be increased more

  We gave a few tasks to the participants to check the users’ efficiency, effectiveness, and satisfaction. The tasks given to the participants are listed below:

  • 1. Could you please go to the learn reading letters and read out letter “S”?

  • 2. Could you please go to the learn reading number and read out numbers “2”?

  • 3. Could you please go to the learn reading number and read out numbers “1” and “5”?

  • 4. Could you please go to the learn reading number and read out words “Ant” and “Cat”?

• Results

  • 1. The animation is not visible.

  • 2. The numbers and letters are visible. The users mentioned it would be nice if they could increase the size of the buttons by themselves.

  • 3. Users suggested adding the guide for connecting the Braille display with the smartphone

  • 4. Users suggested adding dark mode in the application to adapt it easily.

5 Discussions

• Target Audience: Visually impaired and blind people are the primary audience for the application. Although, the application can be used for learning Braille by ordinary people as well.

• Opportunities: The application is unique such that all the features of the application are merging to provide a stage for visually impaired people to learn how to read in a faster way at their own pace. The most astonishing thing about the application is that it is integrated with a Braille keyboard that helps users learn and feel simultaneously.

• Future Scope: The current application helps the users to learn how to read Braille letters. They can even feel the dots by their fingertips using the Braille display. The application consists of three functionalities, of which the first functionality has been developed. The scope for the future of this project consists of the wrest two functionalities: “Practice Reading Letters” and “Practice Reading Words.” The use cases for these functionalities are already being created. The requirements for these functionalities are React Native Developer, Braille Keyboard, Microsoft Azure SQL database. An AI function that can assist users in talking to the app to get to a specific letter they wish to study is another feature that may be introduced to the app. Users will have greater freedom due to this functionality, and the application will be more feasible. This functionality can be implemented using “speech to text” AI technology.

6 Conclusion

To conclude, the developed application has provided a platform for visually impaired and blind people to read the letters from ‘a to z’ and feel the braille dots in the braille keyboard. The application helps them to grow their knowledge and allows them to learn at their own pace. As discussed in the long term, the application will develop further and have more advanced features. It will stand out from other applications as it is integrated with the Braille keyboard, unique. The application is built with React Native, an open-source framework that is entirely free to use. Microsoft Azure SQL cloud data storage is being utilized for the initial data storage, which may be expanded in the future to hold more complex data when the application is developed further. Since the Braille Focus 40 keyboard integrates the application and can be only connected to IOS devices, therefore, for now, the application can only be installed on IOS devices such as iPhones and ipads.