Keywords

1 Introduction

Sign language is a visual gesture-based language used by deaf and mute people for their conversation [1]. Sign Language consists of manual and non-manual [15] parameters. Manual parameters [18] include hand shapes, orientations, locations, straight movements, circular movements. Non-manual parameters include eyes-gaze, shoulder movements, body movements, head nods, face expressions, lips movements, etc. Every word has its sign. As with spoken languages, there are numerous varieties of sign language [12]. But all sign languages are not same all over the world because it changes from region to region, nation to nation and country to country due to cultural and geographical variations, historical context, environmental facts, etc. Each nation or country has its sign language. For example, the British Sign Language (BSL) and the American Sign Language (ASL) are quite different and mutually unintelligible, even though the hearing people of the United Kingdom and the United States share the same spoken language. Sign Language is a complete and natural language. Every sign language has its signs and grammar rules correspond to a word or sentence [13]. It is estimated that more than 200 sign languages are available which are used by deaf and mute people in the world [20].

2 Dictionaries of Sign Language

A lot of work had been done for the construction of sign language dictionaries all around the world. These dictionaries were created for SL in a variety of countries, including China, Korea, the United States, and Vietnam. Real human videos or 3D computer-generated avatars were used to create these dictionaries. Suzuki et al. [21] had constructed a multilingual sign language dictionary for three gesture languages which consist of Japanese gesture Language, Korean gesture Language, and American gesture Language. On the system, the user can control the speed of sign language motion and select the targeted language. Fuertes et al. [6] developed bilingual dictionary for two languages which include Spanish Sign Language - Spanish (DISLE). This dictionary is available in electronic form for the hearing impaired community. The system provides two options to users for word search either in signs or Spanish [bilingual sign language dictionary]. Franc Solina et al. [19] developed a multimedia sign language dictionary that translates Slovenian text to Slovenian Sign Language (SSL). The multimedia dictionary has included 1800 individual signs in the form of video clips. The words that are not presented in the sign language dictionary or the name of persons are represented by finger-spelling [11, 16]. This dictionary was not only useful for reference purposes but also useful for learning in special educational institutions for hearing impaired people, as well as for normal people, who want to learn sign language. Cormier et al. [5] prepared a BSL Sign Bank dictionary for BSL. It includes 2528 words of signs in the form of human video clips. All the sign videos were recorded by the deaf using BSL. Martin et al. [14] constructed an ISL dictionary for disaster warnings. The messages were traced to translate them into 2D avatar animations. The dictionary gives information related to the disaster domain to deaf people in sign language. It includes 600 sentences and 2000 words related to disaster messages. Goyal et al. [8] developed an ISL dictionary using synthetic animations. The dictionary contains 1478 words signs in the form of 3D avatar animations.

3 Techniques and Methods

Hamburg Notation System (HamNoSys) was developed by Siegmund Prillwitz in 1984 at the University of Hamburg, Germany [10]. It is a phonetic transcription system based on stoke notation that can be used for other sign languages. This sign writing notation was developed for the construction of SL dictionaries, machine conversions. It has included 200 symbols which are covering all hand shape gestures as signing parameter that holds hand shapes, orientations, locations, and movements of hand [22].

3.1 eSIGN Editor Software

The eSIGN software consists of two components: the eSign Editor and the JASigning Player [2]. The eSIGN Editor program was developed by the eSIGN project team to allow us to create gestures in the forms of HamNoSys code and convert it to the SiGML script [24].

3.2 SiGML

SiGML is a Signing Gesture Markup Language that is used to represent HamNoSys symbols [3]. The HamNoSys symbols are described in the form of an XML scripting tag for each word. It provides many tools to display the sign of input HamNoSys code in the form of animated symbols and animated videos for communication purposes.

3.3 Synthetic Animations

Avatars are represented as ‘Virtual bodies’ to display input [7]. HamNoSys [23] notation for sign language is readable by three-dimensional (3D) executing software. It takes input as a SiGML file or XML tag and generates output as synthetic animations or avatars corresponding to the input word or text. Animations frames are given as input to the avatar in a sequence for each inputted text. When these frames are placed in a sequence of poses, then rendering software produces synthetic sign animations [17] corresponding to the specific frame definitions. Synthetic animations can be generated in real time reducing the time [9] and storage requirements to a large extent.

3.4 Construction of Proposed ISL Dictionary

The proposed steps for construction ISL dictionary is depicted Fig. 1.

Fig. 1
figure 1

Steps construction of ISL dictionary

Full size image

  • English Text: The system takes English text as input. English word can be days of week, month name, basic phrases, dishes etc.

  • HamNoSys Code: This module convert English word into HamNoSys code by following the structure rules of HamNoSys tool.

  • SiGML code: This module converts HamNoSys code into SiGML code by following SiGML conversion rules.

  • Synthetic Animations: At last SiGML Player plays the SiGML file and generates synthetic animation corresponding to SiGML script file.

4 Comparative Analysis of Current System with Existing Systems

Various existing systems are available to represent sign language. Around the world a lot of work has been done to represent sign language worldwide. Several existed systems have been constructed in the form of pictures which has become oldfashioned. In the modern era of digitization and computerization, Some existed systems have designed using human videos but it required lot of time for recording and space for storage. We designed our current system using synthetic animations that required limited space for storage and time for execution. But it does not looks natural as human videos (Table 1).

Table 1 The total number of words implemented
Full size table

For the accuracy purpose these signs are matched with ISLRTC videos. These signs are also verified by ISL interpreter and the students of deaf schools. Some signs screenshot like show, keep, and happy shown in Table 2.

Table 2 Sample of words implemented
Full size table

5 Result and Discussion

We developed bilingual English text to ISL dictionary using synthetic animations. Unlike other spoken languages such as English, French, Hindi Punjabi, Tamil, etc. sign language cannot be in written form. HamNoSys is a notation system that can be used to write any sign language. This notation system can represent both manual and non-manual [4] parameters of signs. The eSIGN editor software is used to covert HamNoSys code to SiGML code. Then SiGML player generates animations by using the SiGML file. Currently, we have a collection of 3051 words that are used in daily life. 2748 words are transformed into HamNoSys symbols from this list. Name of person and unknown words are finger-spelled. The words are classified under the categorization such as alphabets, numbers, noun, verb, pronoun, interrogative, preposition, etc. are shown in Table 1.

6 Conclusion

In the current task, English words to ISL dictionary has been constructed for Indian gestures using various resources such as ISLRTC (Indian Sign Language Research Training Center) dictionary videos and some videos were recorded by expert ISL interpreters. Using computer generated avatar technology is much superior than using real human video because it consumes less storage space, takes less time for translation, SiGML files can be easily uploaded and downloaded without any delay and the virtual character customised to the user’s preferences. The dictionary can be used in a translation system that converts English text into gesture language and can be used in deaf schools for education. In the future, more signs will be added to extend this dictionary.