Keywords

1 Introduction

The smart phone is becoming an important aspect of our lives. The comfort and convenience certainly makes our lives much easier than ever before. Smart phones really bring so many advantages to mankind. For example, e-commerce connects people everywhere and etc.

Although the smart phone brings many advantages to mankind, it is still difficult for visually impaired or blind people to use the smart phone. Application developers should include functions and features that are able to support blind people. For instance, text-to-speech technology, sound of a clicked button, vibration and etc.

This mobile application is built to help the visually impaired or blind people to identify objects and its characteristics by scanning using a technology called Near Field Communication (NFC tag). This advanced technology requires less efforts to successful identify an object compared with other scanning mechanisms such as QR code. In other words, NFC brings convenience to different kinds of user regardless if they are visually impaired or sighted people.

In this project, the user is allowed to add or change contents with speech-to-text technology unlimited times until the tag is physically damaged. Content is stored in NFC tags independently. No matter if the smart phone is in the online or offline environment, it is still able to access the content of NFC tags. Once a user inserts content into NFC tags, the smart phone will automatically generate and display the contents without opening ant third-party application.

The NFC tag is a passive item. It operates without the need of a power source. NFC tags are powered through a process called electromagnetic induction. As the phone is placed near to the tag, the phone creates a magnetic field and the NFC receiving coil will produce a match with the electrons coming from a powered phone.

2 Existing Systems

2.1 WayAround Application

The WayAround app and the smart WayTags lets the user add helpful information to items around their home and office [1]. The simple tag-and-scan system lets the user add a custom description or more information such as expiration dates of food or washing instruction for clothes.

After tagging an item, scan it anytime with a smart phone to hear the description and details. WayAround use the technologies of NFC (Near Field Communication) in this application. WayTag are designed in 4 types, such as button, sticker, clip and magne [1].

WayAround makes information that has never before been available to people who are blind easily accessible. It is an elegant solution, which meets previously unmet needs of many who live with vision loss [1]. Strengths of this application include adaptability. Waytags have 4 types of tags such as sticker, magnet, button and clips so the user can tag anything and everything. Waytags are reusable. WayAround is affordable, each of the tags cost about a dollar each. Besides that, it is accessible because this application works with NFC technologies to store information and transmit it wirelessly [1]. All a user needs to do is hold the device close to the target WayTag.

The weakness of this application is that it only works with a smart phone that has the feature of NFC (Android phone). iPhone users should install the additional NFC application and it only allow the user to read NFC and cannot insert or modify content inside NFC tags [1]. Another limitation of NFC is it only works in short distance, in a range of 10 cm to read the NFC tag.

2.2 Digit-Eyes

There are 2 versions of this application, Digit-Eyes with full features and Digit-Eyes Lite with limited features. This is an application only available on iPhone, Digit-Eyes allows user to read Digit-Eyes QR (Quick-Response) text bar code labels [2].

They have their own website called Digit-eyes website, which allows a user to create a PDF file of Digit-Eyes QR bar codes that each contain up to 250 character of text. These codes are useful for labelling items where the content that cannot be changed and also create labels to identify clothing, CDs, canning and personal items [2].

Digit-Eyes includes a feature that enables a user to identify many groceries, CDs, and other consumer goods by scanning the UPC and EAN codes on the products. It will connect to the product database, after a matched result is returned in text form. If a user’s phone has Voiceover Active, it will read aloud to the user [2].

The cons of this application are UPC and EAN codes features are only available on the full version application. The user needs to pay USD9.99 to use the full features version. Besides that, this application only supports the iPhone operating system. Based on the chart from [3], there are only 22.17% IOS users in globally, which means this application only focusses on this 22.17% of a narrow market.

The strength of this Digit-Eyes application is that it supports multiple languages including English, Danish, French, German, Italian, Norwegian Bokmal, Polish, Portuguese, Spanish, Swedish [2]. Besides that, it can create its own label code and record information that the user wants in text form and audio form. After the user successfully recorded the information, a rescan of the label code will display the information that the user recorded.

3 Layout Interface Requirement

3.1 Interface Design Requirements for the Moderately Visually Impaired User

Touchscreens became a standard feature of mobile devices such as smart phones and tablets. Touchscreens provide friendly and easy to use interface for modern mobile devices [4]. However, visually impaired people are faced with numerous difficulties using a touchscreen. There are several functions that are able to improve accessibility on smart device, including screen reading, voice input, large buttons, screen magnification and high-contrast screen [4].

Nowadays, the mobile device is designed with the function buttons of numerous items integrated into the software interface, this approach tries to minimize the production costs, but it also increases the difficulties faced by visually impaired user when they experience the touchscreen-based smartphone. Therefore, the researcher should do further exploration on the accessibility of the touchscreen interface and also focus on visually impaired user accessibility requirement and expectation [4].

There are several factors which influence accessible touchscreen interface design. The first factor is ‘Tracking’, which includes bold borders, touch clues and etc. Visually impaired users often collect information by touch, therefore traditional mobile phones with bumps on the keys can enhance their touch perception of an object’s surface, while perceiving objects on touchscreen-based smartphone is challenging for a visually impaired user because the touchscreen surface is flat and slippery. Thus, adding a bold border around the tapped button will help moderately visually impaired user to easily locate and know related clues on the touchscreens. Therefore, providing useful tracking clues for touchscreen is crucial [4].

Second factor is ‘Detection Quality’, this factor helps moderately visually impaired users to perceive signals and other information on the screen. Items including information sequence arrangement on screen, screen density, vibrotactile, reading glass and sensitive click on screen are able to assist the visually impaired user to receive more information. For example, picture and text information should be in big size and arranged [4].

Furthermore, third factor ‘Fuzzy Field’ includes designing a related object shape that easily recognizes the dynamic effects of the item [4]. This factor is mainly for moderately visually impaired because they have some light perception and able to vaguely identify the shape of an object. For these kinds of user, although the information may be not very clear, but fuzzy object shapes able to give them a reference to guide them the right location to press on the screen.

Factor 4, ‘Dominance’ is used to control the capabilities of touchscreen interface, such as voice control and speed of sliding screen [4]. Some of the modern smartphones provide voice control functions. For example, Apple product (Siri) and Android product (Google Now). This is a useful assistance for user with physical disabilities. They can easily interact with smartphones by using voice control to perform daily tasks such as phone calling, listening to music and etc. Moreover, the sliding speed must be adjusted to ensure that the touch feedback was good.

Besides that, factor 5 ‘Colourity’ is another design factor which should be considered for moderately visually impairment user [4]. A high-contrast screen can be result in easy perception of an object, visually impaired people prefer appropriate screen colourity when they are looking for information on the touchscreen. For example, black colour background with text in white colour is the best colour contrast for visually impaired user.

Factor 6 ‘Speech Quality’ refers to the quality of screen reader and natural voice item [4]. Visually impaired users rely on auditory information to perform different mobile activities. When a user press on a block of text or information, the content must be automatically read out. Besides that, the button should announce its function, for example home button should pronounce ‘home’ after user press on it. Screen reader enables the visually impaired user to receive messages through sound, this will increase the speed of receiving information and also reduce the false selections on the touchscreen.

3.2 Interaction Experience of Visually Impaired People with Assistive Technology

Affect from auditory sense play a significant role for visually impaired user [5]. But on this experiment, it was observed that problems included speech is too slow, screen reader voice are not friendly and so on. Screen reader is a crucial element for visually impaired user interact with smartphones, therefore, to develop a good quality screen reader is necessary to meet the expectation of the visually impaired user [5].

Besides that, participants found it difficult to perceive spatial information through an auditory channel. For example, the screen reader expresses the information in nine sections of a screen (3 X 3) (e.g. top left, right down). Many visually impaired indicated that using clock metaphor such as direction 11 o’clock would help them easily to understand spatial structure [5].

Moreover, before beginning to access any application, the visually impaired user will memorize the location of the item or button on the screen by practicing alone or asking to someone help. However, the lack of the tactile landmark on the touchscreen is difficult for them to memorize the interface layout [5].

Furthermore, the most influential factor that affect the ability of moderately visually impaired people to use a smartphone is eye fatigue. Color contrast and transparent background are the factors that cause eye fatigue for people with low vision. Most participants preferred using black background with white text and also suitable text sizes [5].

From 4 experimental findings of unique interaction experience of visually impaired user mention on above paragraph, it is very important for the designer to understand what technologies is able to assist the visually impaired user [5].

First, it is important for visually impaired users to receive information mostly through the auditory sense. Several auditory design features can be included, such as softness of bell sound, number of default bell sounds, and steps required to adjust the volume of the bell [5].

Secondly, visually impaired user cannot estimate distances between items on the screen. Therefore, the experimenter indicated that visually impaired user preferred guidance with the clock metaphor when receiving spatial information on the touchscreen. In addition, development of a navigation system can be helpful for them perceive visual-spatial information [5].

Third, provide a consistent and simple-structured UI layout. From the experiment observation [5], they found out that users tried to memorize the layout of the smartphones before starting to use it. The layout must be consistent and should not change according to the function. For example, (in Fig. 2) the button location inside pop-up components change frequently according to the message length. In other words, the button is not always at the same location. Hence, it was difficult for visually impaired user to know the exact location of the button (e.g. okay, cancel). In addition, the layout of application should be simple and familiar. Visually impaired users prefer a familiar layout [5].

Fourth, increase configurable settings. There should be an additional settings function which can improve the accessibility of application. Text size and colour should not be set arbitrarily because visually impaired users with different levels of severity have different requirements. Providing changeable color contrast, transparency of background, font shape, space between letter and lines will be helpful for them [5].

Fifth, improve speech performance of the screen reader. There is a lack of studies on improving the speech performance for assists blind people [5]. Visually impaired users rely heavily on the screen reader to receive all information. Therefore, the demand for speech performance should be further studied and applied [5].

3.3 Designing Mobile Application for Visually Impaired People

Visually impaired user faces different problems due to the rapid development of information technology has made to mobile applications. Application without features of screen readers, screen magnification, voice commands recognition is useless for visually impaired user [6]. Besides that, accessibility to the functionality of mobile devices is another problem. Modern smartphones and tablets have flat surface of the touchscreen. The touchable “home” button is not enough for a fluent navigation through the content of the screens. Next problem is visually impaired users are impossible download available mobile application in Google Play or App Store. Action such as selecting item from a list or cancellation in the pop-up dialog windows are difficult task for them to do [6].

Most important feature of operating system accessibility for blind people is screen reader. It interacts with user by using various gestures. For the example, a single-tap reads the label of the selected elements. Double-tap activates a selected element, similarly to double-click a mouse button on computer [6].

One of the most powerful features of mobile accessibility is the function of intelligent voice assistant. Mobile personal assistant helps user to answer question, make recommendations. Additionally, it analyses user’s behavior, based on their search queries, it will predict what user may want to know, or what to do. Intelligent assistant is useful for visually impaired user because this the best method to interact with smartphones [6].

Currently most mobile operating system allow user to change text size. IOS 8.1 offers 12 sizes to choose from horizontal slider, but the changed sized only affects e-mails, contract and text messages. While Android OS 5.0 offers 4 different sizes and selected text size applied in the whole operating system. Besides that, both mobile operating system provides color inversion as the option of accessibility. The is a very important option for reducing reflects and visual fatigue. This could be helpful for those visual impairment users. This feature changes the text in black color and white in background [6].

Both IOS and Android operating system have their own built-in screen magnification. This accessibility feature is important and useful for those people with low vision. The IOS includes the ability to increase the viewable display with a three-finger double-tap. With Android OS, activate the screen magnification by triple tapping the screen with a single finger. Disabling magnification is done by triple tapping the screen again with a single finger [6].

4 Proposed System

4.1 Interface Design

The proposed system was implemented by using software Android Studio. Latest version of Android Studio support many different programming languages such as Java, C+ + , GO and Kotlin. This application was built by using Java as the programming language. Java is the official language for programming Android apps, so it could be the best languages for Android apps and there are many useful resources on internet that can be used for references during implementing this application.

Besides that, this application required an enabled NFC smartphone to run and test usability of application. Huawei P10 is the model of smartphone that used in testing and running this application.

Before start to build this application, study interface layout requirement for visually impaired user is a must. After studied literature reviews [4,5,6], learned knowledges was applied during designing interface layout.

Refer to Fig. 1, the insert content process was designed by using the recording mechanism rather than using the QWERTY keyboard, because visually impaired users face challenges to use QWERTY keyboard. Speech-to-text mechanism is used in the recording process. The recorded content will be stored in text form. Besides that, this application also provides a replay button used to check the content if it is captured correctly. After the user confirms that the content was captured correctly, the user is required to place the phone close to the NFC tag in order to successfully save the content into NFC tag. The range of adding content into the tag is within 3 cm distance. If the phone cannot detect any NFC tags, the screen reader will interact with user and tell the user to get closer to the NFC tag.

Fig. 1
figure 1

Implemented application layout

Full size image

In addition, designed button can be single-clickable and double-clickable. The screen reader will speak out the button feature at a single-click, while double-click is used to select the button.

Moreover, this application provided explanation pop-up dialog use to explain the usability of this application. Screen reader will read aloud the content and after the content read finished, the dialog will automatic close. Furthermore, configurable settings were provided in this application to improve the accessibility of application. Users can change the language of the application to English, Chinese and Bahasa Malay, user also allow to change the voice reader.

5 Proposed System

This test was conducted to study accessibility, usability and satisfaction about the implemented mobile application. 30 people were tested, which included sighted participants who were wearing a blindfold and also near-sighted users without wearing their glasses. Participants will be classified into 2 groups, group A is for those participants aged under 30, while group B is for participants aged 30 or above. Participants were given assistance if they had problems. They were required go through the tutorial part first before they start performing any task.

Secondly, they need to change the application language and screen reader type on setting page. Third, they were required to add new content into the NFC tags by using speech-to-text technology, all the processes were estimated to be completed within 15 min. After they finish the ordered tasks, they were required to give their feedback about this application. Several questions about testing usability were prepared as shown in Table 1 to find out whether this application meets the expected result and allows the user to feedback their experience after they run through everything on this application. Likert score scale is used from 1 (Very poor), 2 (Poor), 3 (Neutral), 4 (Good) to 5 (Very good) as illustrated Table 1.

Table 1 Interview questions about usability of application
Full size table

6 Result

After conducting the testing of 2 groups of participants and collecting feedback from the users, result can be referred to in Table 2.

Table 2 Result of interview question (Q3 to Q5 for moderately visually impaired user only)
Full size table

Group A average score of teaching process is 3.04 (Neutral), Screen reader quality in average score of 3.66 (Neutral), font size average score of 4.33 (good), color-contrast average score of 4.33(good), button’s icon average score is 3.66 (Neutral) and overall.

experience average score is 3.83 (neutral).Group B average score of teaching process is 2.5 (Poor), Screen reader quality in average score of 2.66 (Poor), font size average score of 3.66 (Neutral), color-contrast average score of 4.66(good), button’s icon average score is 3.66 (Neutral) and overall experience average score is 3.66 (neutral).

Through this test, we found out that this application still has a lot of room for improvement. Besides that, we retrieved a lot of feedback from users for future improvement. For instance, feedback from participant stating “Explanation is too complicated, if able to change sound effect better and the explanation is not complete”. From this feedback, we learned that if the explanation takes too long, the user will feel impatient to listen to the explanation till the end. The ‘explanation’ feature can be made simpler and easier to understand.

7 Conclusion

The growth of mobile technology is going rapidly in the last few years. As visually impaired or blind users are increasing in this market, it is necessary for the developer to build useful applications for them. Therefore, this system was proposed to enable the visually impaired or blind user to identify object correctly in an effective way and also easy to use way. This mobile application is built by using Android Studio.

The main goals of this project are to provide a simple and easy access mobile application which allow blind and visually impaired users to identify objects quickly and effectively using NFC technology.

Last but not least, after conducting usability testing, users’ overall feedback is neutral, which means that this application still needs to be improved. This application could be improved in the future. For instance, the help function could be improved by making the explanation simpler and placing explanations at every page separately instead of putting all the explanations in one page. Furthermore, more options may be included in the settings page. For example, user can turn on or off the optional sound effects.