> IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. 6, NO.

4, DECEMBER 2011<

28

Assistive Technology for People with Hearing/Speaking Disabilities in Qatar

Ohood Al-Amoudi, Rouda Al-Kuwari, Sara Qaffaf, Nada Aloraidi, Heba Dawoud, Muneera Al-Marri, Tarek El-Fouly, Amr Mohamed
AbstractAs the number of people with hearing and speaking disabilities is increasing significantly in the world, building bridges of communication between Deaf and Hearing communities and the different authorities and institutions has become essential to deepen the mutual cooperation in all aspects of life. The problem could be summarized in one question: How to construct this bridge to allow people with hearing and speaking difficulties to communicate? This project suggests an innovative framework that contributes to the efficient integration of people with hearing disabilities with the society by using wireless communication and mobile technology. This project is completely independent. Unlike the existing solutions, it depends on an extremely powerful Automatic Speech Recognition and Processing Server (ASRPS) that can process speech and transform it into a text. Independent means that it recognizes the voice regardless of the talker and the characteristics of his/her voice. On the other side, there will be a Text To Speech (TTS) engine, which will take the text sent to the application server and transmit it as speech. The second aim of the project is to develop an iPhone/iPad application for the hearing impaired. The application facilitates the reading of the received text by converting it into animations constructed from a database. It also assists the writing process by developing a customized user interface for the Deaf to communicate efficiently with others. Index Terms Assistive Technology, Automatic Voice Recognition, Sign language Animations, Mobile Communication

I. INTRODUCTION

he community with Hearing or Speaking Disabilities represents a significant component of the society that needs to be well integrated in order to foster great advancements in the society. This Project targets people with hearing or speaking disabilities. It introduces a wireless communication solution consisting of three parts. The first part transforms the speech into a text using an Automatic Voice Recognition (AVR) instead of a Relay Service Provider (RSP) assistant, which is independent. The application provides another feature, which can display the transformed text as animated American Sign Language (ASL). At the other end, there will be a text to speech transformation. This will help these individuals communicate in their daily lives with their surroundings and correspond independently using their mobile phones. For example, if a person with Hearing or Speaking Disabilities (HD/SD) wants to contact the ambulance, he will not be able to communicate with them via the phone, nor via an SMS because emergency numbers and other services do not support SMS. Therefore, it is significant for HD/SD to obtain information and receive help anywhere at all times. We are currently using American Sign Language for its simplicity. Nevertheless, the application can be further developed in other languages like Arabic sign language and British sign language. The application also assists the writing process by developing a customized user interface for the Deaf to communicate efficiently with others using a customized keyboard. II. PROJECT SIGNIFICANCE AND OBJECTIVES This project addresses a critical societal problem and introduces an innovative framework that has both social and strategic impact on the way of communication amongst a well-integrated community of the disabled and other people in the society. A. Social Impact This system can potentially be leveraged by mobile service providers and can be offered as a service for Hearing/Speaking disabled people to interact and communicate with others and which will have a profound impact on the way this community communicates. The project helps HD/SD people to have private phone conversations and communicate with services that dont support SMS, which might cause a problem when they want to contact the ambulance in case of an emergency. Usually people with Hearing/Speaking disability depend on SMS, which is not

Manuscript received September 15, 2011. This work was supported in part by Qatar University CSE Department and Qatar University Wireless Innovation Center. O. Al-Amoudi: Computer Engineering Student at Qatar University. Doha Qatar. (e-mail: [email protected]). R. Al-Kuwari: Computer Engineering Student at Qatar University. Doha, Qatar. (e-mail: [email protected]). S. Qaffaf: Computer Engineering Student at Qatar University. Doha, Qatar. (e-mail: [email protected] ). N. AlOraidi: Computer Engineering Student at Qatar University. Doha, Qatar. (e-mail: [email protected]). H. Dawood: Computer Engineering Student at Qatar University. Doha, Qatar. (e-mail: [email protected]). M. AlMarri: Computer Engineering Student at Qatar University. Doha, Qatar. (e-mail:[email protected]). T. ElFouly: Assistant Professor of Computer Engineering and Computer Engineering Program Coordinator in the Department of Computer Science and Engineering, University of Qatar, Doha, Qatar. Phone: (+974) 4403 4248 (email: [email protected]). A. Mohamed: Assistant Professor at Department of Computer Science and Engineering, University of Qatar. Phone: (+974) 4403 4273 (email: [email protected] ). Publisher Identification Number 1558-7908-2011-21

1558-7908 2011 IEEE Education Society Students Activities Committee (EdSocSAC)

http://www.ieee.org/edsocsac

> IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. 6, NO. 4, DECEMBER 2011<

29

considered a real-time communication because it requires time to respond. However, using this application will save time by being able to communicate on the telephone. B. Strategic Impact The application could also be used for other purposes in the future. It can potentially be used for any activity that requires real time translation such as watching and interacting with a television program, listening to a lecture or a seminar, or even a real conversation. C. Objectives Establishing two-way communication between HD/SD and H/S people. Supporting seamless communication calls, i.e. H/S person should be able to talk normally as in an ordinary call. Developing VoiceXML grammar consisting of verity of English words. Creating animations for American Sign Language consisting of letters and words. Developing an iPhone/iPad [10] application with a robust and efficient user interface.

both parties know Sign Language. It is informal for business calls. Additionally, Effective communication depends on the network performance and the quality of the video. The proposed project framework will help people with HD/SD and H/S people to communicate with each other. A phone call can work in both ways, either from an H/S person to a person with HD/SD or vice versa. The first case is that an H/S person will speak on the phone. The voice will be converted into a text which will be received by the other end party of the call, and for a better understanding, the text will be translated into Sign Language and displayed on the screen. Then the reply will be in a text that will be transformed back into speech at the other end. The second case will be similar to SMS but with the extra feature of having the text translated into Sign Language. B. Related Work 1) Automatic Voice Recognition (CapTel) This technology used by CapTel provides a captioning service to its customers whereby a deaf person uses a CapTel telephone to make or receive a call. The CapTel service automatically connects to a captioning service assistant who repeats everything the other party is saying into an Automatic Voice Recognition system which sends the converted text to the CapTel phone along with the voice, and then the Deaf person responds by talking [1]. Hamilton has provided a Mobile CapTel Application for smart phones such as BlackBerry, iPhone, and Android. The concept of the speaker-independent AVR system is used with their Mobile CapTel service. If a person wants to call a person that uses the Mobile CapTel App, first he/she needs to dial the Hamilton Mobile CapTel Center number and then the number of the deaf person they want to call [4]. Both CapTel and Mobile CapTel Application differ from our project. The first difference is that our project uses an AVR system that consists of a Speech Recognition System (SRS) that converts spoken words into a text regardless of the speaker, while CapTel uses a Voice Recognition System (VRS), which must be trained to a particular speaker. Secondly, our project is targeting HD/SD people, which is broader than targeting people with just hearing loss as in CapTel. The last difference is the usability of our project because caller does not need to dial the server to call the other end HD/SD. This means no extra step is required and so any caller could dial-up without knowing that the other end is an HD person. In fact, such functionality would require the intervention of the mobile service provider to route calls according to the user preference to trigger the AVR engine to facilitate the text to voice communication. 2) Signing Avatars (Human like Avatars) The technology of signing avatar has developed to an extent where software programs are able to animate avatars to do a Sign Language from a text like Simon, UK or a voice like SiSi by IBM. Simon is an extremely detailed signing avatar that translates texts from Television captions into signs. The system reads the captions and matches them with its dictionary of motion-

III. BACKGROUND AND RELATED WORK A. Background Before inventing the Telephone Typewriter (TTY) in the 1960s [3], people with HD/SD could not use the ordinary telephones. Over time, this device has been called Telecommunication Device for the Deaf (TDD). A TDD device has a keyboard and an LCD screen which shows that a text could be sent to/received from other TDD devices. A TDD allows HD/SD people to communicate with other people, but it requires both ends to have the same device. Then Telecommunications Relay Service (TRS) was introduced. This service allows TDD users to make phone calls to ordinary phone users with the help of a Relay operator that types everything the Hearing or Speaking (H/S) person is saying and sends it to the TDD [3]. After that, the Caption Telephone (CapTel) was developed using Relay Service but with Automatic Voice Recognition Systems (AVR) that transforms what the Relay operator is hearing and saying to the system into a text, which is sent to the CapTel user device. Relay Services were not limited to any device; the service can be reached using laptops, mobile phones and other devices. With the availability of cameras on these devices, Video Relay Services were introduced (VRS) which allow HD/SD people to use their native language (Sign Language) instead of texting [1]. Other possibilities people with HD/SD have on using regular mobile phones are SMS, Instant Messaging (IM), video calls, and emails. Texting is the most preferred method of communication among them after video calls although it does not provide real-time communication. Video calls are useful if

1558-7908 2011 IEEE Education Society Students Activities Committee (EdSocSAC)

http://www.ieee.org/edsocsac

> IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. 6, NO. 4, DECEMBER 2011<

30

capture data (not videos or images) that includes hand, face and body data that allows Simon to translate the data into animations in real-time [7][8]. SayitSignit (SiSi) is a developed application by IBM that uses a VRS called ViaVoice that converts speech into a text, which is then used to generate the motion of the avatar via a Sign Language dictionary [9]. Both Technologies use signing avatar which will be very helpful because Sign Language is expressed by using hand, face, and lip movements while in this project the animation will be restricted to hands only which limits the created Sign Language animation database. Simon uses the motion-capture while an animation database with video format is used in this project.

Clients HD/SD
Animation module Database module Telephony service Server connection module

H/S

HTTP

CCXML

File.php <?php>

HTTP

Database

Grammar <grammar> <rule> <one-of> <item>..</item> <item>..</item> </one-of> </rule> </grammar> Speech Application

TTS
.VXML

ASR

DTMF

IV. PROPOSED SOLUTION A. Solution Overview

PHP server

Voice Engine

In this section, the main functionalities of the projects system (Refer to Figure 1) will be explained from a high level perspective. The implementation of these functionalities will correctly determine the successfulness of his project towards achieving its goal. These functionalities are: 1) The establishment of a two-way communication phone calls and HTTP sessions between the H/S and HD/SD person, with the ability of initiating the call by either ends. 2) During a phone call conversation, the H/S persons speech should be passed to an AVR engine to be converted into a text and then sent to a web server, which then can be sent to the application used by HD/SD person to be displayed as a text and the corresponding ASL animations. 3) The ability for the HD/SD person to respond by texting back via the application using a customized keyboard through which the text will be sent to a web server that passes it to a TTS engine that will convert it to speech. 4) The ability for an H/S or an HD/SD person to hang up the phone, which will make the system terminate and release the call/HTTP sessions. This solution will be implemented using wireless communication and mobile technology. It will also offer an operator-independent call with complete privacy. The available solutions are all dependent on an operator based information delivery like CapTel [1] in the United States or RNID Typetalk [2] in the United Kingdom. Our solution will be achieved by replacing the operator with a powerful Automatic Voice Recognition engine (AVR). This engine is completely speaker independent that can process speech and convert it into a text with a capability to recognize the voice regardless of the H/S person and the characteristics of his/her voice and whether the speaker is a man, a woman, or even a child. In addition to that, the AVR engine can recognize a large scale of vocabulary.

Figure 1 High Level Architecture The primary methods that will be used to achieve the project functionalities are explained in the following points: 1) Speech Recognition and Speech-to-Text conversion The AVR engine contains two modules: a TTS module that converts a text into speech, and an ASR module that recognizes speech. Besides, a VoiceXML code will be uploaded to run on the AVR engine. When an H/S person calls an HD/SD person, a call session will be initiated and then routed by the service provider to the AVR engine. A specific VoiceXML application will be triggered using Call Control XML (CCXML). Both the VoiceXML application and the AVR engine module (ASR) will cooperate to recognize the speech, store the text in a string variable, and send it to a PHP file that is hosted in the web server, which will submit it to the client (HD/SD person). 2) Display the transformed text as animated American Sign Language Many technologies are provided in the market to meet the needs of people with HD/SD. These technologies could be one of the following: a. A Mobile Phone b. A PDA or a Smart Phone c. A Laptop, a Tablet or a Netbook In this project, the first two technologies will be used with an assistant of the developed application. A text will be received from a file on the web-server and then displayed into an application running on an HD/SD person's device. Simultaneously, the application will search for the animation in the database that corresponds to the received text starting by matching the words. If a word animation is not found in the database, it will be displayed by ASL letters instead. The

1558-7908 2011 IEEE Education Society Students Activities Committee (EdSocSAC)

http://www.ieee.org/edsocsac

> IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. 6, NO. 4, DECEMBER 2011<

31

3) Text-to-Speech conversion The sent text from the client side will be received by the web server, and then passed to the VoiceXML application, which will prompt the text. B. High level architecture The components of the high-level architecture are illustrated in Figure 1. The functions of these components will be explained in the coming paragraphs. HTTP: In order to increase ubiquity and promote ease of deployment, the HTTP protocol will be used as the main application protocol for communication between the three system components. In HTTP the sending component acts as a client and the receiving component acts as a server. In case that the H/S person starts the phone call, the AVR engine acts as a server and the PHP application that hosts the communicating text acts as a server. On the other hand, the PHP application acts as a client when it sends the text to mobile application. CCXML: Call Control XML - is the standard language for controlling how phone calls are placed, answered, transferred, and more. CCXML works hand-in-hand with VoiceXML to provide a 100% standards and XML based solution for any telephony application. [11] This interface is used to route the phone call to the intended VoiceXML application. Therefore, it connects the person who placed the phone call with a proper VoiceXML application that is loaded on the AVR engine. VXML: VoiceXML is a language for creating voice-user interfaces especially for the telephone, It uses the speech recognition and touchtone (DTMF keypad) for input, and pre-recorded audio and text-to-speech synthesis (TTS) for output. [12]
Type with customized keyboard Search for text in DB

C. Developed System 1) User Interface Design The iPad application named Interpreter includes three tab views: Conversation View for the communication, Last Conversation View to retrieve the last conversation, and About View to present information about the application and the developers.

Figure 3 Conversation View The conversation is the main view (Figure 3), which comprises multiple functional elements: 1: Send button to send the message to the caller. 2: Table View such that all sent and received messages are displayed in text. 3: Cell selected or row selected represents the message being displayed in animation. 4: Empty cell. 5: Text View where the user enters text. 6: Keyboard that consists of customized keyboards, one for most common words (Figure 4). 7: Video display of the Sign Language being animated by one hand or both. 8: Show the keyboard button. 9: Disconnect the call button.

Controller
Animate text to ASL Display text Check which button is pressed

View
Display animation Path of animation file

Model

Figure 2 Model View Controller For the iPhone/iPad application development, the MVC (ModelViewController) software architecture is used (Refer to Figure 2) The MVC mainly consists of three components: a) a Model which manages both the data (animation database) and the business/domain logic, b) a View which consists of multiple views to handle rendering of the model into a user interface element that is suitable for interaction, and c) a controller that receives the users inputs and initiates a response based on those inputs [13].

Figure 4 Customized Keyboard of Words 1558-7908 2011 IEEE Education Society Students Activities Committee (EdSocSAC)
http://www.ieee.org/edsocsac

> IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. 6, NO. 4, DECEMBER 2011<

32

2) Functionality A call session established between the H/S person and the HD/SD person will be routed to an AVR server. Hence the VoiceXML application installed on the AVR engine will be triggered and an automated response from the voice engine will ask the user to say a message. The message is then transferred to the iPhone Operating System (iOS) application (Interpreter) via the established socket connection and represented as Sign Language animations concurrently with the message. The animations could be repeated upon user request when the messages cell is selected (key number 3, Figure 3). After that, the person using the application (Interpreter) types a response using the keyboard and sends it. The response is delivered as post request (text) to the PHP server to be forwarded to the AVR engine and transformed to voice. When the conversation is over, the user will be able to terminate the call from the application by closing it or pressing the disconnect button (key number 9, Figure 3). On the other hand, the H/S person can terminate the session by saying no to the repeated VoiceXML script question Do you want to continue?. V. CHALLENGES Some of the various challenges we have faced in this project: First, the Automatic Voice Recognition system may not understand the persons pronunciation; the person should speak clearly in a proper English language with a certain accent. Communication challenges: Critically, in this part of the project we encountered significant challenges on which we tried different approaches to solve them. At the beginning, the communication with the PHP server was done over a File Transfer Protocol (FTP) server using text files. Uploading and downloading text files was the first approach. However, we found it insufficient to use this method in Network Programming. Therefore, we sought a better alternative: sending post and getting requests. The post request method was done perfectly. Conversely, our idea about the get request was inadequate. In Network programming, requesting (get a request) every 0.5-seconds is insufficient to receive a text from the PHP server. Therefore, we obtained the last approach, which is using socket connection to receive the text from the PHP server. Database challenges: The full-fledge database of the whole sign language was not available; consequently, we started with a limited dictionary of most common words reading the data from the database continuously. Hence, the database files created on sqlite3 needed to be open. The solution was to create and check the database to open it with the launch in the application. Continuous speech Challenges: At the beginning of this project Voxeo platform was used, in the self-learning process. However, when the design of the project was started, VoiceXML application, Loquendo was chosen since it was recommended for having the capability to recognize the continuous speech without specifying the sentences or phrases on the grammar. It was found out that it is not possible for the Loquendo ASR module to recognize the speech - a sentence the caller says. It must be identified in the grammar to be

recognized by the ASR module. However, this is not a requirement because it is impossible to guess all the possible phrases the caller may say during a call session. This problem was resolved using a special structure for the grammar.

Figure 5 Grammar Structure The grammar should have three main compounds as shown in Figure 5: Vocabulary: The vocabulary contains all the possible words that the caller may say. Complex rule: The complex rule calls the vocabulary, and uses the repeat property that is used to call the vocabulary more than one time depending on the number of times that was specified in the repeat option. So if <item repeat="1-3"> is written, this mean that the user should say minimum one word and maximum 3 words from the vocabulary. In this case, when <item repeat="1-"> is written, this means that the caller has to say at least 1 word. However, the ASR manual states that the number 33 is the default maximum number of repeats if it was not specified. Public rule: The public rule calls the complex rule, and when the complex rule returns what the caller says it stores it in the initialized field. Limited Grammar challenge: The VoiceXML Grammar Database was not available. Therefore, after insuring that the grammar is able to recognize continuance speech, we prepared a list of 72000 different English words to improve the accuracy of the voice engine. However, the maximum amount of words accepted by the application reached 650 words. Consequently, the size of grammar was reduced. Keyboard challenges: We tried to use the iOS keyboard and enhance it by adding some word keys to the existing letters, numbers, and symbols. However, this could be done by jail breaking the iOS to gain full access root access to unlock its features which was totally unacceptable. Thus, creating multiple sub-views [14] to build a customized visual keyboard was the best approach to encounter this challenge. Challenges in animating the sign language: We faced some problems in choosing the appropriate software for the

1558-7908 2011 IEEE Education Society Students Activities Committee (EdSocSAC)

http://www.ieee.org/edsocsac

> IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. 6, NO. 4, DECEMBER 2011<

33

animation subsystem of the project. This problem was solved quickly and we were able to create acceptable amount of animations as videos for different words and letters using 3D Studio Max 2010. Nevertheless, other problems were encountered while developing the application itself. The most difficult challenge was to play the videos animated hand gestures continuously without showing the transferring between the two consecutive videos to the HD/SD. Also, creating two symmetric hands in the software was challenging because the purchased hand model was not well designed therefore, some modifications were necessary to be done to satisfy the need. The mobile application interface in general has to be highly customizable and relevant for people with Hearing/Speaking Disabilities to simplify the use of the application for communication. VI. TESTING VOICE ACCURACY By this test, the accuracy of the voice server and its ability to recognize the speech of the callers is measured. Every volunteer is asked to use the testing application. It asks the user to talk. Then the application says the recognized phrase. The volunteers are asked to say four different phrases with different lengths. The accuracy of the voice engine is tested based on two main categories: gender and nationality in addition to the length of the phrase used in both categories to acutely analyze the accuracy. The following results show the correlation of the nationality, phrase length, and accuracy testing. Figure 7 Accuracy for Arabic Speakers The results of the accuracy testing shown in Figure 6 and Figure 7 illustrate that the voice server can perfectly recognize the speech of non-Arabic speakers even for long sentences. However, the voice server recognizes the sentences in an acceptable way for Arabic speakers. It misses some words and recognizes others, since it is highly dependent on the accent of the caller. Figure 8 and Figure 9 demonstrate the results of the gender and phrase length testing taking into consideration the same nationality. One can conclude that there is no significant difference between the gender results. The voice engine can clearly recognize short sentences better than long ones, regardless of the callers voice characteristics.

Figure 6 Accuracy for non-Arabic Speakers

Figure 8 Accuracy for Female Arabic Speakers

1558-7908 2011 IEEE Education Society Students Activities Committee (EdSocSAC)

http://www.ieee.org/edsocsac

> IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. 6, NO. 4, DECEMBER 2011<

34

VIII. CONCLUSION Bringing to a conclusion, the primary aim of this project is achieved by establishing a two-way communication between the hearing and speaking disabled person and the able one. Sending a text from a person with hearing/speaking problems to be received as a voice to the hearing/speaking person and vice versa succeeds by sending post requests and socket connection. Moreover, other options are added such as retrieving the last conversation and the ability to either choose just reading the received text or animating it constantly on the iPad. ACKNOWLEDGMENT Upon the completion of this project, we would like to express our gratitude to Qatar University Wireless Innovation Center for their valuable support. Special thanks to Dr. Adnan AbuDayya, Mr. Hani Belal, Mr. Mohammad Hasan and Mr. Taha Bebek for sharing their knowledge, suggestions, moral support and patience to guide us throughout our work. We are also thankful to everyone who participated and helped us from Qatar University namely Dr. Ryan Riley and our supervisors: Dr. Tarek and Dr. Amr. We also appreciate the support from prof. Qutaibah Al-Malluhi and the Department of Computer Science and Engineering. REFERENCES
[1] Captel. Available: HTTP://www.captel.com/ Date visited October 12, 2010. [2] Text relay the UK's text to voice relay service. (2009). Available: HTTP://www.textrelay.org/ Date visited October 12, 2010. [3] Mackenzie, A. (2010, April 15). The history of TTY for the deaf. Available: HTTP://www.ehow.com/about_6300381_history-tty-deaf.html Date visited November 1, 2010. [4] Placing a call using the hamilton mobile captel iphone app. (2010). Available: HTTP://www.hamiltoncaptel.com/mobile_captel/iphone_app/placing_ca lls.html Date Visited October 20, 2010. [5] Textrelay help page. (2010). Available: HTTP://www.ukecc-services.net/Textrelayhelp.cfm [6] Ahmed, A, & Seong, D. (2006). Signwriting on mobile phones for the deaf. Proceedings of the 3rd international conference on Mobile technology, applications & systems, Retrieved from ACM Digital Library Date visited December 10, 2010 [7] Halawani, S.M. (2008, Jan). Arabic sign language translation system. IJCSNS International Journal of Computer Science and Network Security, 8(1), 251-256, Available: HTTP://paper.ijcsns.org/07_book/200801/20080136.pdf Date visited November 20, 2010. [8] Kim, S, Li, Z, & Aoki, Y. (2004). On intelligent avatar communication using korean, chinese and japanese sign-languages: an overview. International Conference on Control, Automation, Robotics, Vision, 1. from IEEExplore Digital Library Date visited November 18, 2010. [9] Say it sign it : converting speech to deaf signs. (2010). Available: HTTP://www.sys-consulting.co.uk/web/Project_SISI.html Date visited December 10, 2010. [10] Apple Inc. (2010) Available: http://www.apple.com, 2010. [11] What is CCXML. (2010). Available: HTTP://www.voxeo.com/library/ccxml.jsp Date Visited October 20, 2010.

Figure 9 Accuracy for Male Arabic Speakers VII. FUTURE WORK We can add many advanced features in the application for future work such as: 1) Making this application global determines the international usability by appending different languages. This will be a great improvement for the project. 2) Enhancing the animations part by adding an avatar to be able to change the facial expressions that will improve and simplify the hearing/speaking disabled persons understanding of the message. It could be implemented by using an advanced camera that captures movements and takes 3D images to view the facial expressions precisely, i.e. using the Xbox Kinect. 3) Adding well-known vocabulary words and slang to the list of words in the database. 4) Adjusting the font and the animation size through other user convenience options. 5) Saving the wanted conversations in a history tab instead of the last-conversation tab via additional settings. 6) Adding the user contacts in the application for direct dialing. 7) Choosing a theme for the app that satisfies the HD/SD. 8) One of the objectives was assisting understand what others are saying in the same room for Sign language Translation aimed at people in direct communication. However, its not been implemented yet and we are planning to do it in the near future. 9) Compatibility could be added by scaling the mobile application on multiple platforms. Sharing this application with other platforms such as Android, Blackberry, Nokia and the new Windows mobile would satisfy all users. 10) Using push notifications to inform the HD/SD person about the received calls when the application is running on the background.

1558-7908 2011 IEEE Education Society Students Activities Committee (EdSocSAC)

http://www.ieee.org/edsocsac

> IEEE MULTIDISCIPLINARY ENGINEERING EDUCATION MAGAZINE, VOL. 6, NO. 4, DECEMBER 2011<
[12] Voicexml tutorials. (2010). Available: HTTP://www.palowireless.com/voicexml/tutorials.asp Date Visited November 20, 2010. [13] Modelviewcontroller. (2010). Available: HTTP://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93co ntroller Date Visited November 25,2010. [14] Brant, T. (2010). iPad & iPhone app development. New York: Alpha.

35

Ohood Al-Amoudi graduated with Honors from Qatar University with a Bachelor of Science in Computer Engineering. Rouda Al-Kuwari graduated with Honors from Qatar University and received a Bachelor of Science in Computer Engineering. She is currently working in Q-CERT, ictQATAR. Sara Qaffaf graduated with Honors from Qatar University with a Bachelor of Science in Computer Engineering. Nada Aloraidi received her B.Sc. with Honors in Computer Engineering in 2011. Heba Dawoud obtained her Bachelor of Science in Computer Engineering at Qatar University. Muneera AlMarri biography was not available at the time of publication. Tarek El-Fouly is a faculty member in the Department of Computer Science and Engineering at Qatar University. Amr Mohamed is a faculty member in the Department of Computer Science and Engineering at Qatar University.

1558-7908 2011 IEEE Education Society Students Activities Committee (EdSocSAC)

http://www.ieee.org/edsocsac