Machine Translation Approach To Translate Amharic Text To Ethiopian Sign Language
Machine Translation Approach To Translate Amharic Text To Ethiopian Sign Language
Abstract
Sign language is a communication system using gestures that are interpreted
visually. Many people in deaf communities around the world use sign languages as
their primary means of communication creating a strong sense of social and cultural
identity. About 0.4% of the whole population of Ethiopia is deaf and out of this,
more than half is illiterate because of the fact that there are very limited studies
conducted on Ethiopian Sign Language. This research document presents a Machine
Translation system to translate Amharic text to an equivalent Ethiopian Sign
Language (ESL) by finger spelling representation with the help of a 2-dimensional
animating avatar rendering the equivalent ESL finger translation. Animating an
avatar for rendering Ethiopian finger spelling requires a dynamic and periodic
switching of hand shapes and hand movements. Macromedia Flash 8.0 and
ActionScript 2.0 are used as a tool to model and design the avatar, the hand shapes
and the hand movement definitions. The translation system is tested by 10 deaf
people and 66.6% of them responded that the overall performance is good and
supportive.
Introduction
According to Ethiopian National Association for Deaf (ENAD)
referring Ethiopian national statistical agency's study, about 1.5 million deaf
people lived in Ethiopia in 1994 E.C. and latter in 2000 E. C., the deaf
252
population raised to approximately 2.5 million. About 8% of the whole
population of Ethiopia is disabled people. Out of these, 2% counts for deaf
and hard of hearing people.
Deafness, characterized by visual communication, creates a strong
sense of identity among the society of any country. Visual communication
like sign language is the primary means of communication among the deaf.
Except for parents of deaf people, teachers of deaf students and people
interested on sign language; most of the hearing society does not know sign
language. This created a huge communication gap between hearing people
and the deaf. The researcher believes that a research attempt on translating
either from the auditory language (Amharic) to that of sign language (ESL)
or the vice versa eases the communication gap. There is either two-way
translation (Bilingual translation) or one-way translation among natural
language translation systems (Emiko, n.d.). Bilingual translations are
commonly observed for languages of same nature (spoken/written
languages). A translation made between sign languages and spoken/written
languages like Amharic is more manageable if accompanied in one-way
translation approach because of the nature of the two languages. In one-way
translation, the translation direction is best identified based on previous
research attempts, available resources and the worth of beneficiaries. In the
case of Amharic and ESL, there are two directions of translation: from
Amharic to ESL or from ESL to Amharic. Such translations are conducted
considering that ESL is of the deaf side and Amharic is of the hearing
society. Translating Amharic to ESL benefits the deaf because it helps the
deaf understand what is stated in Amharic. It also helps the Amharic
speakers express what they want to say to the deaf. Translating ESL to
Amharic on the other hand, benefits the hearing people because it helps the
hearing people understand what is said in ESL.
254
motivated symbols to describe these manual and non-manual features of
signs (Prillwitz, Leven, Zienert, Hanke, Henning, et al, 1989).
Visual languages systems especially like sign language are expressed by
combining different components of the sign. Signs have the following
components (Daniel, 2006) (Kadous, 1995)
I. Hand shape
Hand shapes are usually used to represent finger alphabets of sign languages.
Hand shapes are created by bending and stretching the finger and the palm.
There are signs created using only one hand like Ethiopia, America and two
hands like British.
II. Location
Location refers to the place where the signing is performed. The signing can
be performed either by hand, eye or other body. There are signs that are the
same with other components of sign and only defers in their location. For
example, signing the word "mother" and the word "father" in ESL have the
same hand shape except for the sign for "mother" is signed locating hand at
the fore-head and that of "father" at the chin.
III. Movement
There are a wide variety of movements possible with sign language
including arcs straight lines and wavy patterns. Hand movement is
dominantly used to make the major component of ESL. Head movement, eye
movement and the torso movement may also be applied in ESL.
IV. Orientation
There are three possible orientations of the hand: roll, pitch and yaw.
Finger Spelling
Finger spelling is a manual representation of written language, and it
is used as a substitute for speech as a live, or face-to-face, medium of
communication. It is a means of communication rather than spelling of any
255
spoken language words. It is used in combination with sign language for
proper nouns, names and addresses and for words that have no specific sign
(Riekehof, 1987). The term finger spelling stands for both sign language
production and perception (Evans, 1982).
When finger spelling, people use their dominant hand to make hand shapes,
one corresponding to each letter of the word. Fluency in sign language
includes the ability to produce and recognize finger spelling at a rate of four
characters per second (Wolfe, 2004).
256
Ethiopian Finger Spelling
ENAD (Ethiopian National Association for Deaf), has developed the
Ethiopian Finger Spelling in 1971(Legesse, 2008). Latter, Ethiopian Finger
Spelling was accepted and recognized by the Ministry of Education.
Currently, it is used in all deaf schools along with the American finger
spelling. The American Finger Spelling is used when we want to express
words or concepts in English and local languages based on Latin alphabet,
like Oromiffa, while Ethiopian finger spelling is used when we need to
express words or concepts in language
anguages that use Gee’z alphabet, like
Amharic and Tigrigna (Legesse, 2008). Ethiopian Finger Spelling has 34
unique hand shapes to represent the first order Amharic alphabet
alphabet, where 33
y’) have been used until 2009 (Legesse,
of them (except the Amharic letter ‘y
2008). The hand shape for letter ‘y’’ (shown in Figure 2) is lately introduced
as the 34th finger spelling of ESL by ENAD. Figure 1 shows the 33 Ethiopian
finger spelling with the corresponding Gee’z alphabet except letter ‘y’ as
found from ENAD.
257
Figure 2: the lately introduced letter ‘y’ (v)
258
research, the phonetic mode (P) is selected to take Amharic input from
keyboard because the phonetic mode is the simplest and common for people
who do not type Amharic frequently. The researcher believes that the
simplicity may help both the deaf and hearing people in easing the
application of this new study results.
Concepts Data Systems P.L.C presented the phonetic mode of Gee’z
alphabet in such a way that it corresponds with the English sound of that
letter by English alphabet, as presented in Table 1.
Consonant u I A y e o
Ó°´ Ÿw° XMe ^w Ue dÉe dw°
Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Group 7
259
No
movement
Ó°´ Ÿw° XMe ^w Ue dÉe dw°
Consonant u i a y e o
Figure 3: Amharic vowels associated with ESL finger spelling hand
movement.
Translation architecture
This research investigates the conversion of Amharic text into the
equivalent ESL finger spelling sign. ESL finger spelling translation needs an
“heuristic” approach that can be mapped in the direct and transfer paths of
Machine Translation architecture. These are defined in the experimentation
part of this study as methods of hand-shape modeling and hand movement
definitions. The pyramidal model of machine translation architecture is
customized in this research as in Figure 4.
260
Interlingua
Direct translation
261
Amharic text input
Break into
consonant and vowel
Unicode
reference
has
vowel?
No Consonant
Integrate
Hand movement shape and
definition movement
Hand shape
definition
Output
No
262
When an Amharic text is typed from the keyboard, the Unicode
equivalent of each letter is collected from a module. Note that the Unicode
equivalent of each base letter of Gee’z alphabet is defined in a module for
the purpose of displaying the Amharic font at the input box. Each letter of
the input text is then broken into a consonant (group1) and vowel-containing
(Non-group1) category and stored in an array. Here, if a letter is categorized
as vowel-containing, it is identified by the base letter and the following
vowel. For example, in the word “S`"„”, the phonetic mode key sequence
becomes “m, re, ka, to”. Only the first letter “S” is characterized by a
consonant category (group1). But the remaining letters are all characterized
by a vowel-containing category (Non-group1). For instance, “`” is
phonetically typed as “re” where “r” represents the base letter “[” from
group1 and the vowel “e” is appended to it. The following table illustrates
the consonant / vowel category on the above example.
S m (S) Group1
` r ([) E Non-group1
„ t (}) O Non-group1
Translation algorithm
The translation algorithm shows a step-by-step transition of the
translation process passing through each component.
Step 1: Enter an Amharic word by using the phonetic mode of power Gee'z.
Step 2: If unsigned character is available:
pick one character and identify the character's behavior based on the two
categories, Group 1 letter or none group 1 letter.
Sub procedure:
If the input letter contains either of the 6 vowel sounds (a, u, i, a, y and o),
then it is identified as vowel containing (none-group 1).
Otherwise, it is a group 1 letter.
Step 3: If the letter is group 1, then it picks the correct hand shape for the
hand shape library. Otherwise, pick the correct hand shape for the hand
shape library and refer the right type of hand movement corresponding with
the attached vowel.
Step 4: Integrate the hand shape information and the hand movement
definition and provide- input it to the avatar.
264
Step 5: Go to step 1.
265
Macromedia Flash’s symbol type that can be manipulated and called by an
action script language). The resemblance sample between the avatar’s hand
at design mode and the real picture of the letter “u” is shown in figure 7.
267
Hand movement definitions are defined based on the avatars two
joints: the elbow (Hinge joint between the forearm and upper arm) and the
shoulder joint (joint between upper arm and shoulder). Flexible rotational
operations on these two joints suffice to show the 6 different hand
movements for group2 though group7. First, a default rotational angle is set
to 00 with a reference of the vertical avatars standing direction for both joints.
Vertical and horizontal referencing is frequently applied to the rotational
operation to result on the right hand movement. Additional operations like
scaling and translation (movement) are slightly applied to create virtual
realism.
Elbow
Rotation with
horizontal reference
Stick picture shows the rotational operations applied on the elbow of the
avatar’s hand with respect to the vertical and the horizontal references.
269
Table 4: Statistical Illustration of the Respondent on the first 4 Objective
Parameters
Parameter Options Respondents
Frequency Percentage
Conclusions
ESL lacks a standard writing system and besides very limited studies
has been conducted on ESL finger spelling, despite the fact that it is quite
270
necessary to help the existing communication gap between hearing people
and the deaf. Animating any character is a challenging task, especially when
the animation is humanoid and intended to act as a human. Avatar
construction is quite different from other animations because, avatars need to
have some more heuristics of human operations.
In this research, an attempt is made to adopt a machine translation
approach to translate Amharic text into ESL finger alphabet word. The
machine translation architecture is adopted from Interlingua system of the
three-level Machine Translation Architecture. The adopted architecture and
the translation model are designed in such a way that it can be a good input
for any researcher who want to improve this system or even to construct ones
own avatar on other platforms or other media.
Recommendations
A number of shortcomings can be observed from this research that
may initialize further similar studies. The following issues need to be
considered to complement the study area.
• It is quite convincing to use avatar based machine translation
especially for Sign language related researches. From the literatures
reviewed in this research and performance analysis, it has been observed that
using a 3D avatar can definitely better convey sign languages than 2D
avatars.
• ESL is complete in both signing and finger spelling. Signing includes
the conceptual sign expression which is dominantly applied to convey
meaning in ESL. Hence, there is a need to study a Machine Translation of
conceptual ESL expressions with a high level heuristic-based animation
technique. It is even better if both, the finger spelling and the conceptual
expression, could be incorporated together in a single application.
271
• This research reads only a single word to translate in to ESL finger
alphabet expression which is not applicable for a phrase, sentence, or a
document translation. Therefore it is necessary to extend the system so that it
handles sentences or even a document.
• Identification and classification of nouns and action words (verbs) of
ESL is worth studied issue because ESL or any sign language use finger
spelling fully or partially for nouns and proper nouns. On the other side, sign
languages use of conceptual expression for action words or verbs. This
would ease the translation of Amharic text to ESL or other sign language.
Reference
Augustine Su Project. (n.d.). Accessed October 1, 2002, from
http://www.csdl.tamu.edu/su/asl
Augustine Su Project. (n.d.). Retrieved October 1, 2002, from
http://www.csdl.tamu.edu/su/asl
Becky Sue Parton, Sign Language Recognition and Translation: A
Multidisciplined Approach from the Field of Artificial Intelligence,
2004
Daniel Assefa: Amharic Speech Training for the Deaf, Masters Thesis, Addis
Ababa University, Computer Science Department, 2006
Dorr, P. Jordan, and J. Benoit. 1998. “A Survey of Current Paradigms in
Machine Translation.” Technical Report LAMP-TR-027, Language
and Media Processing Lab, University of Maryland
Emiko SUZUKI, Mariko H., Koyoko K. Bilingual Sign Language (n.d.):
Dictionary for Learning a Second Sign Language with out Learning
the Target Spoken Language
Endale Asefa, Design and Development of Hypermedia Learning Tool for
Ethiopian Sign Language,2005
Ethiopian National Association for Deaf: BERTAT (Yearly Magazine).
Addis Ababa: Ethiopia, 1997
Evans, L.: Total Communication: Structure and Strategy. Florida: Gallaudent
College press, 1982
Francik, J., & Fabian, P. (2002). Animating sign language in the real time.
Paper resented at the 20th IASTED Conference on Applied
Informatics, Innsbruck, Austria.
H. Kamp and U. Reyle. 1993. From Discourse to Logic Introduction to
Model theoretic Semantics of Natural language, Formal Logic and
272
Discourse Representation Theory. Kluwer Academic Publishers,
Dordrecht.
Ian Marshall, E´ va Sa´fa´r, A Prototype Text to British Sign Language
(BSL) Translation System
Kadous, M. W.:Recognition of Australian Sign Language using
Instrumented Gloves, Honours Thesis, School of Computer Science
and Engineering, University of New South Wales, 1995
Kyle, J : Sign and School: Using sign in Deaf child’s development. 1987
Liddell. 2003. Grammar, Gesture, and Meaning in American Sign Language.
UK: Cambridge U. Press
Matt Huenerfauth, English Text into American Sign Language Animation,
Computer and Information Science University of Pennsylvania
Philadelphia, PA 19104
Morford and J. MacFarlane. “Frequency Characteristics of ASL.” Sign
Language Studies, 3:2.
Sáfár and I. Marshall. 2001. “The architecture of an English-text-to-Sign-
Languages Translation System.” In G. Angelova, ed., Recent
Advances in Natural Language Processing. Tzigov Chark, Bulgaria.
Sandler, W. : An Overview of Sign Language Linguistics. In Keith Brown,
(ed.), Encyclopedia of Language and Linguistics. 2003.
273