Introduction

Linguistic diversity is an imminent feature of India. The languages spoken in India belong to different language families, yet the majority of them belong to the Indo-Aryan language family. Marathi is one of the spoken languages that belong to this family. It is the official language of Maharashtra state (India) and is the 3rd most spoken language in India. On the other hand, Indian sign language (ISL) is used by the hearing-impaired community all over India with some regional dialects. It has a formal grammatical structure.

Though there are ample natural language processing (NLP) applications available, the languages they can process is still a subset of existing (spoken as well as sign) languages used worldwide. Many languages are still in need of NLP tools and computationally feasible resources that can be utilized by existing NLP approaches and applications. Marathi and Indian sign language are both among them. This language pair deficit a compatibly large bilingual corpus that is sufficient to develop a data-driven or knowledge-based translation system. Thus, a rule-based machine translation system for translation from Marathi to ISL can be a stepping stone to enhance these circumstances.

According to the 2011 Indian census, there are 1.3 million people with “hearing impairment.” The World Health Organization (WHO) data cite around 63 million people suffering from significant auditory loss. Still, there was always a bias in India about the use of sign language. But circumstances started improving after India signed the ’UNCRPD’s Rights of Persons with Disability Act,’ in 2016. The awareness about the use of ISL is rapidly increasing since the last decade.

Currently, sincere efforts are being taken by the Indian Government, special schools, and numerous NGOs to use ISL. Indian Government has released a video-based dictionary of ISL signs in three volumes. But most of the resources use pre-recorded videos that are not dynamic and cannot be used for run-time translation of a spoken language to sign language. This justifies the need for a computer-aided system for translating a spoken language to sign language. Such a system can help hearing people communicate with hard hearing effectively.

In the field of machine translation, both kinds of translation systems, spoken language to sign language and sign language to spoken language, are noted. Such systems are termed cross-modal translation systems as the source, and target languages use different communication modes. The utterance, perception, and (specifically morphological) grammatical features of spoken languages are ’linear.’ But, in the case of sign language, grammar, facial expressions, and the use of signing space add extra dimensions to the signing sequence. In sign languages, many important linguistic and grammatical features are simultaneous. This key difference between source and target language creates exciting challenges for machine translation.

We are making efforts to translate simple Marathi sentences to equivalent ISL representations. Our foundational work for the development of a phrase-based translation from Marathi to ISL is presented in [1]. It presents the essential morphological and syntactical features of Marathi and ISL in a contrasting way. The system architecture and distinguished results are also illustrated.

During the exhaustive linguistic study of ISL, it is noted that ‘simultaneous morphology’ is a distinguished feature of ISL. It is imperative to uphold the same during its generation for any NLP application. The article reports an extension to earlier work where we have focused on the processing of simultaneous morphological features of ISL during the generation phase. It is arranged as follows. A literature review on spoken sign language machine translation systems is in Section “Literature Survey”. Section “Sign Formation and Computational Representation” introduces phonetic details about sign formation and its computational representation. The instances of simultaneous morphology (specifically in ISL) are described in Section “Instances of Simultaneous Morphology in Indian Sign Language” in detail with appropriate examples. Section “System Architecture” describes system architecture. In Section “Results and Illustrative Examples”, the results of the framework are discussed using appropriate illustrations. The details about the underlying database and experimentation are inscribed in Section “Experimentation”. Section “Evaluation” encodes the evaluation of the framework. A comparative analysis of our efforts with existing and notable systems is performed in Section “Comparative Analysis of proposed system and existing systems” followed by concluding remarks in Section “Conclusion”.

Literature Survey

As already stated, the grammatical study of the source and target language has been an essential part of this work. The roots of Marathi can be found in 1000AD [2]. Several grammarians and linguists have documented their history research. Several books and research materials are available for the study of the Marathi grammar framework. Most of the study for this work has been done from [3] and [4].

Concerning Indian sign language, the research work and field work done by Dr. Madan Vasishta [5] followed by Ulrik Zeshan [6] is the pioneer. The grammar analysis of Indian sign language done by Dr. Samar Sinha [7] has helped us to get detailed insights into the target language of our system.

A variety of efforts for spoken-to-sign language translation are noted in the last few decades. Researchers have also made efforts the development of sign language interpreting tools. Some of the imminent systems are discussed in this section. A special note of systems dealing with ISL is made at the end of the section.

The initial systems for spoken-to-sign language translation were rule based. The translation was done either direct, syntactic, or at the semantic level. In 2000, the Zardoz System [8] was developed using PATR-based unification grammar and an interlingua approach for English to American sign language translation [9] employed the TEAM system using a syntactic transfer-based approach. It was initially proposed for English to American sign language; later on, also implemented for South African sign language. Synchronous tree-adjoining grammars (STAGs) are used to create source and target language parse trees. Human modeling and simulation technology is used to generate animation using a 3D graphical avatar. Ian Marshall and Eva S’afar have utilized a semantic transfer-based approach for the translation of English text to British sign language in the visual sign language broadcasting system (ViSiCast) [10].

Probabilistic machine translation techniques require an aligned corpus. [11] has taken efforts for the creation of corpus based on the Airport domain. [12] is an important effort using example-based and statistical machine translation for English to American sign language translation using aligned sentences and chunks. An automatic evaluation was done for annotations or GLOSS. A manual evaluation was done for animation, considering the characteristics like understandability, fidelity, naturalness, etc.

Some systems based on neural machine translation are enlisted in Table 4 and compared with our system.

In the case of Indian sign language, all the noted systems have used a rule-based approach. INGIT [13] has been the very first attempt at the translation of Hindi strings to ISL strings. The domain of the system is based on the communication between a railway reservation clerk and a passenger.

[14] developed a dictionary tool for the conversion of Hindi/Bengali/English to ISL. This tool can be used to associate signs with the words, phrases, or sentences of a spoken language text. The sign associated with each word is composed of its related part-of-speech and semantic senses. In the later phase of this venture, a prototype for English to ISL translation is employed. The system can handle morphological functionalities like discourse, direction, and classifier predicates minimally. Only simple sentences in English can be utilized as input.

[15] has made an admirable attempt for sign language generation system based on ISL grammar. The system is publicly available for translation of English to ISL in the form of a web interface and mobile application. The system uses the Stanford parser to analyze the linguistic structure of input English sentences. This linguistic information is utilized, and then sign language gloss is translated using ISL grammar rules.

Apart from this, the Gurumukhi script to ISL translation system is developed by [16, 17] developed a system that pre-processes Hindi text and translates it into ISL grammar using a dependency parser and WordNet 6.

It is observed from the literature survey that sign language machine translation systems mainly use a rule-based approach may be due to the unavailability of linguistic resources. Very few systems have used a statistical approach using a small corpus. HamNoSys and SiGML are used by most of the systems to generate animation for the interpretation of sign language. Very few attempts are made for the translation of Indian regional languages to ISL out of which most of the systems are domain bounded. We have reported a prototype for Marathi to Indian sign language using phrase-level rule-based approach in [1]. Also, up to the best of our knowledge, none of the machine translation systems generating Indian sign language has reported any dynamic processing for incorporating simultaneous morphological features during the generation phase.

Sign Formation and Computational Representation

Sign languages are gesture-based natural languages. It uses body parts like hands, fingers, shoulders, and face movements for articulation. They have evolved over a while and have a formal linguistic and grammatical structure like spoken languages. A ’sign’ in sign language is a meaningful combination of the following phonetic parameters. A significant change in one of these aspects may alter the sign’s meaning or result in a completely different sign.

  1. 1.

    Manual Features:

    • Hand-shape: the shape of the hand used in the sign

    • Orientation: the direction of the palm and fingers

    • Location: the position of the hand in the signing space

    • Movement: the way the hand(s) move through space

  2. 2.

    Non-manual Features (NMF): Facial expression, eye gaze, movements of the head and body

It is essential to note that sign languages use both manual and non-manual features simultaneously to convey the message. In many cases, the articulation of a sign is done at a specific location in the signing space. The visual signal created by these gestures is multi-dimensional. The visual perception system used in sign language, i.e., the eye, can receive and process this multi-dimensional signal.

Though sign language produces a visual signal, it can be encoded textually. ’Gloss’ is used for the same. ’Glossing’ means to describe something briefly and misleadingly. It is a kind of annotation used to write brief one or two-word translations in one language for a word or morpheme in another language. It can encode morpho-syntactical information in sentences.

In sign language linguistics, ’gloss’ plays an important role [18]. It is sign-language-independent and does not have any universal rules. Linguists can use conventions as per their comfort. It is mainly used to represent ’signing sequence’ along with grammatical, spatial, and non-manual features. Any spoken language script can be used to write a Gloss of sign language. This paper uses a capitalized Roman script for writing Gloss. For example, ’I SCHOOL GO’ means the sign ’I’ is followed by the sign of SCHOOL, and then the sign of ’GO.’

As discussed in the introduction, to encode manual, non-manual, and spatial features concurrently, a three-dimensional gloss structure is devised. Each sign is represented using a triplet (MA, NM, SP) where MA, NM, and SP are manual, non-manual, and spatial features, respectively. The symbol,’–’ is used in the absence of any specific value of the corresponding feature. Following are some examples with explanations.

  • (SCHOOL, –, –): Only manual articulation for the sign of ‘SCHOOL’ without any specific non-manual and spatial features.

  • (EAT, YN, –): Manual articulation for the sign of ‘EAT’ along with facial expressions for closed (Yes–No) questions. No specific spatial feature is used.

  • (TABLE, –, IPSL): Manual articulation for the sign of ‘TABLE’ and it is articulated at the ipsilateral side (IPSL) of the signer to assign that location to the object (i.e., a ‘table’ in this case). No special non-manual feature is used.

  • (GIVE, YN, 1–3): Manual articulation for the sign of ‘GIVE,’ with non-manual expression for a closed question. The ‘1–3’ spatial feature suggests the location for beginning and ending locations for the respective verb sign. As gloss is a textual representation of sign language, it does not tell what signing will look like. i.e., gloss is not able to encode the phonetic parameters of manual features.

For the generation of animation, we have used HamNoSys codes [10]. They encode phonetic particulars of signs. Later, these codes can be converted to sign gesture markup language and animation can be rendered using SiGML player [19]. Figure 1 illustrates this process.

Fig. 1
figure 1

Sign language representation using animation generated by HamNoSys codes and SiGML

Full size image

Instances of Simultaneous Morphology in Indian Sign Language

A ’sign’ in sign language is the smallest meaning full unit, just like a ’morpheme’ in a spoken language. In spoken language linguistics, the morphemes are distinguished as free morphemes (having stand-alone meaning) and bound morphemes (appearing as part of a bigger expression). In turn, bound morphemes are functionally classified as derivational and inflectional morphemes. The bound morphemes are formed using affixation. It is important to note that, in the case of spoken languages, morphemes are usually combined sequentially (either as prefixes or suffixes). E.g., un-believe-able, fruit-full, etc. Certainly, a’ sign’ can be modified for morphological purposes. A sign can be modified in different ways, sequentially or simultaneously.

  1. 1.

    Morphological Reduplication: One sign is duplicated to show repetition of action or pluralization of a noun, e.g., GIVE +  +  + , BOOK +  +  + 

  2. 2.

    Compounding of signs: Two or more signs are concatenated together to form a compound sign.

e.g., BEAUTIFUL = FACE + NICE,

KITCHEN = COOK+EAT+HOME

  1. 3.

    Stem Internal Change: One or more phonetic parameters of root signs are changed.

  2. 4.

    Addition of Non-manual and Spatial Features: Facial expressions are added to the sign to express some grammatical property. Also, the locations of articulation are changed to exhibit grammar properties.

The reduplication and concatenation of signs are a sequential type of change. They are evident and easy to handle computationally. But ’stem internal changes’ and simultaneous addition of non-manual and spatial features make computational processing of sign language interestingly challenging. This section discusses some of such circumstances where one needs to perform such morphological changes in a ’sign.’

The following subsections discuss important, distinguished features of ISL where a ’sign’ is modified for morphological purposes [7]. Representative examples using a Marathi sentence and corresponding gloss are also explained.

Numeral Incorporation

In the process of’ Incorporation,’ a phonetic parameter of one sign is replaced by that of another sign. It results in a complex sign. In ISL, a numerical quantifier is incorporated in time units (like day, week, and year). In this process, the time unit is articulated using numerals’ hand-shape instead of the original hand-shape. The resultant sign creates an adverbial time reference, and it is signed at the beginning of the sentence. Only numbers up to 4 are incorporated into signs of time units. Figure 2 shows phonetic details of the root sign of ‘day’ and ‘week’ using HamNoSys codes and animation frames. It also shows the simultaneous morphological change in root signs due to numeral incorporation. The highlighted portion in the HamNoSys code in Fig. 2 shows how the hand-shape needs to be changed to perform numeral incorporation.

Fig. 2
figure 2

Examples of numeral incorporation

Full size image

Syntactically, in a Marathi sentence, a quantifier followed by a time unit acts as a temporal adverb. It occurs pre-verbal or even at the beginning of the sentence. In ISL, such a phrase is signed along with a tense phrase at the beginning of the sentence. Some examples are given as follows. Note that ‘ = ’ in ISL gloss is used for representing the ‘incorporation’ of two signs.

Example 1

Marathi Sentence: मी दोन दिवसांपूर्वी पुस्तक वाचले. [Mī dōna divasāmpūrvī pustaka vācalē.]

Meaning in English: I read the book two days ago.

ISL Gloss: ((TWO=DAY, -, -) (I, -, -) (BOOK, -, -) (READ, -, -))

Remark: The original hand-shape in ’sign of day’ is replaced by the hand-shape of ’TWO.’

Ad-position Incorporation

This kind of ’incorporation’ is applied where an ad-position follows a noun. Ad-positions like in, inside, under, above, and below are incorporated into nouns. But ad-positions like near, with, etc., do not undergo incorporation. In the case of ad-position incorporation, one hand articulates the noun sign, and the other articulates the ad-position. Figure 3 shows a classic example of ad-position incorporation with its HamNoSys codes and animation frames. The highlighted portion in the HamNoSys code in Fig. 3 shows how the orientation of hands needs to be changed to perform Ad-position incorporation.

Fig. 3
figure 3

Examples of ad-position incorporation

Full size image

In the Marathi language, a locative ad-position follows a common noun. Usually, such a phrase functions as a locative adverb. According to ISL grammar, it appears at the beginning of the sentence. Some examples are given as follows.

Example 2

Marathi Sentence: पक्षी घराच्या आत बसतो.[Pakṣī jhāḍācyā Āta basatō].

Meaning in English: The bird sits in the house.

ISL Gloss: ((HOUSE=IN, –, –) (BIRD, –, –) (SIT, –, –))

Remark: Because of ad-position incorporation, the non-dominant hand articulates ‘HOUSE’, and ‘ON’ is articulated by the dominant hand on the non-dominant hand.

Classifier Incorporation

In sign language literature, the phonetic parameter ’hand-shape’ is used to categorize lexical items into semantic classes. This categorization is based on characteristics like shape, size, and qualities of the object. There are fourteen classifier hand-shapes defined in ISL. Table 1 lists some representative examples of the classifier (hand-shapes).

Table 1 Examples of classifier hand-shapes in ISL
Full size table

In ’classifier incorporation,’ the ’verb sign’ is articulated with a hand-shape specified by the classifier. It is important to note that classifier incorporation is performed only with locomotive verbs (e.g., give, bring, put, pick, etc.). The process of classifier incorporation gives a visual effect as if the signer is carrying the object in the hands. Representative examples of classifier incorporation (for noun books and mugs) are given in Fig. 4. The highlighted portion in the HamNoSys code in Fig. 4 shows how the hand-shape needs to be changed to perform classifier incorporation.

Fig. 4
figure 4

Examples of classifier incorporation in ISL

Full size image

Computationally a locomotive verb having a non-abstract argument is needed for classifier incorporation. The classifier for the argument is incorporated in the verb sign. Following are some examples of classifier incorporation phrases in a Marathi sentence and corresponding ISL Gloss.

Example 3

Marathi Sentence: मी चेंडू ठेवतो. [Mī cēṇḍū ṭhēvatō].

Meaning in English: I put the ball.

ISL Gloss: ((I, –, –) (BALL, –, –) (CL:BALL=PUT, –, –))

Remark: The classifier hand-shape of ‘BALL’ replaces the original hand-shape used in ‘PUT.’

Non-manual Feature Incorporation

In ISL, there are some instances where Non-Manual Expressions are used to exhibit ’mood’ features. A specific ’non-manual feature’ is used along with a ’verb sign’ to distinguish the ’mood’ of a sentence. The ’raised eyebrows, nodded head,’ are expressed on the face for yes/no questions. Conditional sentences are also represented using distinguished NMF. The last sign of a conditional clause is accompanied by ’raised brow and head forward’ followed by a brief pause before the main clause. Figure 5 represents the visual distinction between a declarative sentence and a closed question. The phonetic details are shown using corresponding highlighted HamNoSys codes.

Fig. 5
figure 5

Distinction between a declarative sentence and a closed question using non-manual features

Full size image

In Marathi, a question mark at the end of a declarative sentence optionally preceding by 'का' [ka] or 'ना' [na]is used to represent the closed interrogative form of a verb phrase. The following examples show a Marathi sentence and ISL gloss, distinguishing a declarative sentence from the yes–no question.

Example 4

Marathi Sentence: तू आंबा खातो. [Tū āmbā khātō].

Meaning in English: You eat a mango.

ISL Gloss: ((YOU, –, –) (MANGO, –, –) (EAT, –, –))

Remark: This is a declarative sentence. There is no specific NMF.

Use of Signing Space for Verb Agreement

In ISL, the verb does not agree with the gender and ‘number feature’ of its arguments. But directional verbs (ask, talk, help, give, advise, bring, give, invite, etc.) and spatial verbs (go, walk, run) show agreement with the person feature of arguments. Specific locations in the signing space are utilized for this purpose. It is achieved as follows.

The space in front of the signer where the articulation of hands is performed is known as the signing space. Particular locations in the space are of grammatical importance. The signing space is also used for the localization of non-present participants (third person). They are generally localized at the ipsilateral and contralateral sides of the sign. Figure 6a shows specific locations in the signing space corresponding to person features (1): First-person, (2): Second-person, 3A/3B: Third-person). The onset (a location where a sign’s movement begins) and offset (a location where a sign’s movement ends) agree with the locations (and ultimately with the ‘person feature’) of its arguments. Figure 6 also shows a pictorial example for various forms of the sign verb’GIVE’ according to the ‘person features’ of its arguments. The highlighted portion in the HamNoSys code in Fig. 6 shows how the location of the sign needs to be changed for verb agreement.

Fig. 6
figure 6

Verb agreement in ISL using change in onset and offset

Full size image

The spatial feature in Gloss is used to encode the onset and offset of the verb. The default onset and offset of verb signs are from first-person to second-person locations. For subjects and/or objects belonging to the third person category, their locations are marked by ’upper body tilt,’ ’finger-pointing,’ or ’eye gaze’ towards corresponding locations. Following are some examples of verb agreement in Marathi and ISL.

Example 5

Marathi Sentence: मी त्याला पाठिंबा देते. [Mī tyālā pāṭhimbā dētē].

Meaning in English: I support him.

ISL Gloss: ((I, -, -) (HE, -, 3A) (SUPPORT, -, 1-3A))

Remark: ’HE’ is localized at the 3A. The onset and offset of ’SUPPORT’ are locations 1 and 3A, respectively.

Thus, this section has discussed some prominent features of ISL. The incorporation (of classifier, numeral, and ad-position) and verb agreement using signing space is performed by ’stem-internal change.’ At the same time, the addition of non-manual features and spatial features add one dimension each to the gloss. Section “Instances of Simultaneous Morphology in Indian Sign Language” discusses the system architecture of the framework designed to translate ’Simple Marathi sentence to Indian Sign Language.’ It also mentions how the system tackles the ISL features discussed in this section.

System Architecture

To decide the design strategy of this system, we have done an exhaustive morpho-syntactic analysis of Marathi and Indian Sign Language. This study is described in our work [1] contrastively. This study concludes that the word order of main constituents in Marathi and ISL sentences is the same, i.e., SUBJECT–OBJECT–VERB. But the relative order of other components like adjectival phrases, adverbial phrases, negation phases, and WH question markers is dissimilar. So, we did not find any need for deep parsing of the whole input sentence. The study led us to follow the translation process at the phrase level. The morpho-syntactic analysis has also helped us decide the structure of phrases (to be utilized by translation rules) and formulate translation rules.

At the next level of research, we are now incorporating the processing of various morphological features of ISL (discussed in Section “Sign Formation and Computational Representation”) in the existing translation framework. Accordingly, four modules are designed. The database management module has a Marathi Vs. ISL bilingual dictionary based on the vocabulary needed for daily communication. The ’root form’ of each ISL sign is stored using HamNoSys code and the corresponding SiGML file parallelly.

The pre-processing module initially tokenizes the input Marathi sentence. The PoS categories of all tokens are derived. The vital phrases are then identified using regular expressions. These phrases are utilized by the morpho-syntactic knowledge extractor. Various GNPC features specific to particular phrases are extracted and noted for further processing. Sentence level knowledge like TAM features and sentence structure, sentence type is extracted from the verb phrase. Feature structure is used to encode this information. Refer to Fig. 7.

Fig. 7
figure 7

System architecture

Full size image

Translation rules then utilize the phrases and the morpho-syntactic knowledge. The identified phrases are processed to generate an ISL gloss based on ISL grammar. The three-dimensional gloss encodes the signing sequence in terms of manual, non-manual, and spatial features.

The existing SiGML files define the root signs. The ’dynamic sign modifier’ checks if there is any need to make changes in the root sign. Accordingly, it performs various operations like the addition of non-manual features, change in spatial features, incorporation of two signs, etc. The modified sign is then used to generate the Master-SiGML file; the animation player will play it.

NLTK Toolkit is used for pre-processing input Marathi sentences. The ’LXML’ library is used to handle SiGML file modifications at run-time.

Results and Illustrative Examples

This section discusses the illustrations depicting how the system upholds the simultaneous morphological nature of ISL. The dynamic changes in root signs are done at run-time during the generation phase of machine translation. Table 2 illustrates some input sentences and intermediate results.

Table 2 Illustrative example and intermediate output
Full size table

Experimentation

The system is implemented using the vocabulary needed for daily communication. Currently, the system collectively has more than 1000 words of various PoS categories. Table 3 shows the frequency of words based on their morphological classification. This PoS categorization is done from the prospect of spoken language, i.e., Marathi in this case. But for the processing of morphological inflections in root signs, we needed to classify some PoS categories further according to ISL grammar features.

Table 3 Word frequency in database
Full size table

For the numeral incorporation, the adverbs are further categorized in calendric units as दिवस (Divasa, day), आठवडा (āṭhavaḍā, week), महिना (mahinā, month), वषर् (varṣa, year). The system can successfully identify phrases like दोन महिन्यांनी (Dōna mahin’yānnī, after two months), चार आठवड्यांपूर्वी (cāra āṭhavaḍyāmpūrvī, before four months), etc., and process them successfully for simultaneous morphological inflections. In the case of ad-position incorporation, the system can process various Marathi phrases headed by common nouns and followed by ad-positions like च्या वर (Vara, above, खाली (khālī, under), मागे (māgē, behind), पुढे (puḍhē, in front of), जवळ (javaḷa, near), लांब (lāmba, far).

For classifier hand-shapes incorporation, the common nouns are again classified as per their respective classifier hand-shapes. As classifier incorporation is performed only with a specific set of verbs, the main verbs are further classified into plain verbs, directional verbs, and locomotive verbs as per their semantic nuance. The classifier hand-shape incorporation is tested successfully with locomotive verbs like ठेवणे (Ṭhēvaṇē, to put), उचलणे (ucalaṇē, to pick), आणणे (āṇaṇē, to bring) and directional verbs like सांगणे (sāṅgaṇē,), विचारणे (Vicāraṇē, to ask), उत्तर देणे (Uttara dēṇē, to answer), जाणे (jāṇē, to go). The directional verbs are also successfully exhausted for the inflection due to personal feature agreement with the subject and object.

Various types of Marathi sentences (like assertive, negative, imperative, and interrogative) are designed using the phrases that map to these instances of simultaneous morphology of ISL. The system can successfully identify and process them for the generation of ISL.

Evaluation

There are various evaluation matrices available in the literature. But adequacy and fluency are the most common evaluation matrices suggested for a rule-based translation system. Adequacy represents how much the translation in the target language conveys the original meaning in the source language. In comparison, fluency is related to grammatical correctness and word choice for the translation. The evaluation of the reported system has been done manually at two levels. The ISL gloss is evaluated for adequacy and fluency. The rendered animation is assessed for understandability and fidelity. Understandability is used in concern with the articulation of hands and non-manual expressions.

The evaluation of the system was carried out using the expertise of government-authorized ISL interpreters, and special education teachers in Hearing-Impaired schools. Two ISL interpreters and five special education teachers have contributed through their expertise for evaluation. The evaluators have reported their scores on a 5-point scale. Table 4 enumerates the average of the scores reported.

Table 4 Evaluation matrices and score
Full size table

Comparative Analysis of Proposed System and Existing Systems

We have performed a comparative analysis of our efforts with existing systems based on various parameters. Table 4 shows the outcome. As discussed in the literature survey very few systems have used probabilistic machine translation approach. As we are working with low-resource languages, we have used a rule-based approach because of the lack of sufficient-sized bilingual parallel corpora. As denoted in Table 5, many of the systems have not deliberated the key feature of simultaneous morphology or it has been minimally taken care of. So, the main objective of our work was to develop a module to handle simultaneous morphology for Marathi to ISL machine translation in an inexpensive way. The module design is robust and can be generalized for other language pairs. As per the feedback from evaluators, our results are encouraging.

Table 5 Comparative analysis of proposed system and existing systems
Full size table

Conclusion

Simultaneous morphology is a distinguished feature of sign languages. Our thorough study of ISL revealed that it is significant to embody these features while generating them for any NLP application. We have studied such instances from available ISL literature from a computational perspective.

Our study has led to the design and development of a module that can dynamically make simultaneous morphological inflections to ISL root signs (represented using HamNoSys codes and SiGML). This module is being used in the phrase-based machine translation framework for translating simple Marathi sentences to equivalent ISL representation. The results are promising and prove that the morphologically simultaneous nature of sign languages can be preserved to a large extent while generating dynamic ISL.

The framework is tested using a database based on the vocabulary needed for daily communication. The evaluation of the system is performed manually by ISL experts. There is improvement in the fluency of the machine translation process because of the incorporation of the simultaneous morphological nature of ISL during the generation phase.

Based on the linguistic study as well as efforts taken for Marathi to Indian sign language machine translation, we can affirm that this approach and system architecture can be extended for translation using another source and target language pair using minimum updates. We are also working on the automatic development of bilingual corpora of Marathi to Indian sign language gloss using the developed system. The statistical and knowledge-based approaches can be exhausted using such a corpus.