Keywords

1 Introduction

According to the World Health Organization (WHO), around 466 million people worldwide currently have disabling hearing loss, and around 95% of deaf/hard of hearing children are born to hearing-impaired parents. Based on the estimation of WHO, 900 million people will be suffering from hearing loss in 2050Footnote 1.

Many hearing-impaired people use Sign Language (SL) for communication. SL is a linguistic system, consisting of a structured and organized set of signs, with a precise grammar, syntax, and morphology of its own. However, SL communication is not universal. Differences exist between countries and there are even regional variations of SL [1].

SL is the preferred choice for people who have totally lost their hearing capability. SL users rarely use hearing aids, write notes or lip read. So, SL communication leads to several issues for deaf people when interacting with non-deaf people that cannot understand SL. The development of methods that provide some types of ICT solutions for deaf-hearing and hearing-deaf communication has increased. In order to ease their communication, there is considerable research activity in the development of possible supporting tools. These tools have a wide range from systems of Speech to Text (which translates verbal speech into written words and vice versa with Text to Speech), Display System, Haptic or Visual Feedback (e.g., applications focused on multimodal approach, able to convert speech to visual contexts and vibrations, and similarly, the contexts and vibrations can be converted to speech [2]) up to Mobile support [3].

There are also wearable technologies but, that often means that the deaf person must wear numerous very visible devices (e.g., gloves with sensors, helmet with infra-red filters camera). This could potentially represent a stigma for the hearing-impaired person.

Computer-mediated SL communication involves new ways to manipulate language structure and performance, including experimentation of new proposed language forms. Thanks to the developments of new tools, for the first time, hearing-impaired people can communicate using manual visual language, which in many cases is their native language, across space and time zones [4]. Also, the animation capabilities of modern screens offer an exciting opportunity to create a new communication system for people who use SL to communicate. So, SL communication needs to be further investigated to support it through tools and applications.

This work is focused on a component proposed in a project aimed at developing a remote video application to support SL interpreting. A messaging system prototype that makes use of Augmentative Alternative Communication (AAC) methodology is presented. SL is a movement-based language without a standard written form; because of this, deaf people communicating via SL may face several difficulties, including the use of the written form [5]. In order to support chat communication between SL users, in our prototype, we propose the integration of a gallery of Graphics Interchange Format (GIF) representing phrases and words in SL format to promote written communication closer to the needs of the deaf or hard-hearing people.

2 Design Approach

Communication is one of the most important aspects affecting the social interaction for the deaf. Additional methods such as the use of AAC may be used to overcome communication problems and barriers for deaf people [6]. AAC is an area of clinical practice that seeks to reduce, moderate, and compensate for the temporary or permanent difficulties of people with severe communication disorders both on the expressive and receptive fronts and help them in their social needs of life [7]. For deaf people, one of the possible used AAC-based practices are representational systems [8]. The best known is the Picture Exchange Communication System (PECS), which is a communication strategy based on the exchange of several PECS images that make up a sentence. PECS is used to remedy the mismatch in the communication methods of people with deafness and one or more intellectual disabilities (mild or moderate) who communicate through the SL and those who do not use it. In this way, studies - proposing illustrated dictionaries to facilitate message writing intended for their communication partners - have shown that, with adequate preparation regarding the rules of use, the subjects were able to use these tools with success and they have had the role of facilitating writing [9].

Following the representational assisted AAC methodology, in this work, we propose the use of short and silent moving images, the GIFs with the aim to facilitate text-based computer-mediated communication for the deaf. The idea is proposing a chat component based on the AAC communication system. To prepare the aforementioned GIFs, videos provided by the Spread the Sign’s website were converted through a special app.

3 Chat Prototype

The chat component is part of a remote communication tool designed for those deaf people who use SL interpretation. Similar characteristics and tasks offered by typical popular chat systems for the communication between users are provided by the proposed chat component (see Fig. 1). To support chat communication for the deaf user, in addition to the traditional text editing via keyboard, a series of GIFs is shown when the specific button “GIF SL” is hit. Those GIFs representing sentences and words expressed in SL can be selected by the user to edit the message to be sent via chat.

Fig. 1.
figure 1

Chat prototype window.

Full size image

The GIFs set developed for the chat prototype contains a limited number of GIFs images, but it was enough to test the proposed approach. Therefore, the main limitation is that the user may not find the appropriate SL GIF needed to compose and send a certain word/phrase. On the other hand, the main benefit of GIFs is that they can automatically play short, non-audio “looping” video clips [10]. Thanks to these characteristics, the GIF format is well suited as a representation system for assisted AAC. GIFs, in fact, as non-audio contents, are a very suitable medium for the SL rendering, because they can show SL characteristics like: (1) the area where the sign is made, showing part of the signer’s body; (2) the configuration of the hands in executing the sign, that is the shape it assumes by positioning the fingers; (3) the orientation of the palm of the hand in making the sign concerning the signer’s body; (4) the movement of the hand or hands in carrying out the sign [11]. The GIF format can play animations in an endless silent loop, so it is suitable for learning new signs.

The GIFs gallery layout is in line with the common grid outline used for the AAC systems, in which the symbols are in specific positions within a grid (see Fig. 2). As each symbol is isolated, it supports the recall of the individual contents of each cell [12]. The use of GIFs representing SL movements grouped in a gallery is an interesting PECS-based methodology that can be great benefit to the Deaf community.

Fig. 2.
figure 2

A subset of a SL GIFs gallery

Full size image

4 Conclusions and Future Work

This paper presents a potential assistive technology to support chat communication by the deaf. The chat prototype was designed to be used by deaf people who use SL to communicate and have difficulty with written production. The proposed system differs from existing solutions in (1) no advanced digital skills are required and (2) only a few clicks are required to write SL messages.

The current chat prototype has the sign language limitation which needs to be considered in the future work. In fact, our proposed system has been designed for the Italian Sign Language (LIS), and so it needs to be adapted to the various existing sign languages. So, further future development will have to consider the interface localization so that the gif gallery will be shown in the selected language.

In addition, future studies will further develop the chat tool so that the system provides automatic translation from text messages to GIFs (i.e. to SL) and vice versa, thus also allowing communication between a person who uses SL to communicate and one who does not.