THIS MINI PROJECT WORK IS SUBMITTED IN PARTIAL

FULFILLMENT
OF THE REQUIREMENT FOR THE DEGREE OF

BACHELOR OF TECHNOLOGY

DEPARTMENT OF ECE
ASANSOL ENGINEERING COLLEGE
AFFILIATED TO
MAULANA ABUL KALAM AZAD UNIVERSITY OF TECHNOLOGY
JUNE, 2022
 DEPARTMENT OF ECE
ASANSOL ENGINEERING COLLEGE
Vivekananda Sarani, Kanyapur, Asansol, West Bengal – 713305

Abstract ............................................................................................................. 06

1. Introduction ........................................................................................................07

2. Components ........................................................................................................ 08-12

3. Arduino Program ................................................................................................ 13-17

4. Block Diagram and Circuit Design...................................................................... 18-19

5. Motivation of the project...................................................................................... 20

6. Result & Discussion............................................................................................. 21

7. Future Scope of The Project ................................................................................. 22-23

8. References ........................................................................................................... 24-25

Gesture Vocalizer is a project for social purpose. From the survey we found that it is really very

difficult for deaf peopleto communicate with other people. Normally deaf people speak with

hand gestures and it is difficult for other people to understand their sign language. We are going

to implement gesture-based vocalizer which will detect all the gestures of deaf people and

convert it into voice and also candisplay it on LCD screen. For that we are making use of ARM

7 controllers to interface all of the sensors and speech synthesizer. Basically, data glove

contains two types of sensors flex sensor and accelerometer as a tilt sensor. A wireless data

gloves is used which is normal cloth driving gloves fitted with flex sensors along the length of

each fingerand the thumb. Dumb people can use the gloves to perform hand gesture and it will

be converted into speech so that other people can understand their expression. This system is

useful for dumb people to communicate with other people.

06
 1.

In our daily life we used speech and gestures of hand for communication between human

beings. Research in this project aims to integrate the gesture Human Computer Interaction

(HCI). For communication between human being gestures and speech are completely

coordinated to each other. This paper analyses the data from data glove for recognition of signs

and the gestures. A system is designed for recognizing these signs and their conversion into

voice. The result of this design expects the accurate and noiseless conversion of gestures into

voice. A gesture in a sign language is movement of fingers and tilting of hand in particular

directions withspecific angles. Signs language consists of different signs for differentwords. It

also contains of different sign for different letters to createwords that don’t have any sign in that

sign language. In this device we used Flex sensors which plays a major role in our project. Flex

sensors are used at each finger of hand. When we bend our fingers, it changes the resistance

which in turn converted into voltage. It is the device which contains data glove that can translate

Sign language into speech to make the communication between the deafpeople with normal

people easily. We are using (ARM7) controller for interfacing the communication between deaf

and normal people.We are also displaying the conversion of gestures in the form of texton LCD

display for the people who are unable to hear the voice. So that they can easily communicate

with the deaf people.

07
 2.

A flex sensor is a kind of sensor which is used to measurethe amount of defection

otherwise bending. The designing of this sensor can be done by using materials
like plastic and carbon. The carbon surface is arranged on a plastic strip as this
strip is turned aside then the sensor's resistance will be changed.

Flex sensors plays the major role in Gesture Vocalizer. Flex sensors in gesture
vocalizer is used as bend sensors. We connect the flex sensors at each finger of
hand. Flex sensors are used to detect the bending of fingers.

Fig 1: Flex Sensor

08
 The Arduino Uno is an open-source microcontroller board based on the
Microchip ATmega328P microcontroller and developed by Arduino.cc. The
board is equipped with sets of digital and analog input/output pins that may be
interfaced to various expansion boards and other circuits.

Gesture controlled robot works on the principle of accelerometer which records

hand movements and sends that data to the comparator which assigns proper
voltage levels to the recorded movements. That information is then transferred
to an encoder which makes it ready for RF transmission.

Fig-2: Arduino UNO

09
 HC-05 Bluetooth Module is an easy-to-use Bluetooth SPP (Serial Port Protocol)

module, designed for transparent wireless serial connection setup. Its communication

is via serial communication which makes an easy way to interface with controller or

PC.

In this project the role of Bluetooth module configured only by AT Commands.

Master can initiate connection with other Bluetooth module while slave can receive

data. It cannot initiate the connection with another module.

Fig-3: Bluetooth (HC-05)

10
 An accelerometer is an electronic sensor that measures the acceleration forces
acting on an object, in order to determinethe object’s position in space and monitor
the object’s movement. Acceleration, which is a vector quantity, is the rate of
change of an object’s velocity (velocity being the displacement of the object
divided by the change in time).
Accelerometer in the Gesture Vocalizer system is used as a
tilt sensor, which detect the tilting of the hand. 3.3 Tilt Detection: Accelerometer is
used to detect the bending of thehand and sending the data, to the bend detection
module. Theoutput of accelerometer is obtained by amplification.

Fig-4: Accelerometer (ADXL335)

11
5.

A jump wire is an electrical wire, or group of them in a cable, with a connector or pin

at each end, which is normally used tointerconnect the components of a breadboard or

other prototype or test circuit, internally or with other equipment orcomponents, without

soldering.

In this project the jumper wires are connecting the components and passing the signals to

the component.

Fig-5: Connecting Wire (Jumper Wire)

12
#include <SoftwareSerial.h>
SoftwareSerial mySerial(5,4);
char temp = '0';
//variable initializtion
int xpin = A5;
int xadc = 0;
int xmax = 0;
int xmin = 1023;

int ypin = A6;

int yadc = 0;
int ymax = 0;
int ymin = 1023;

int FLEX_PIN1 = A0;

int flexADC1 = 0;
int sensorMin1 = 1023;
int sensorMax1 = 0;

int FLEX_PIN2 = A1;

int flexADC2 = 0;
int sensorMin2 = 1023;
int sensorMax2 = 0;

int FLEX_PIN3 = A2;

int flexADC3 = 0;
int sensorMin3 = 1023;
int sensorMax3 = 0;

int FLEX_PIN4 = A3;

int flexADC4 = 0;
int sensorMin4 = 1023;
int sensorMax4 = 0;

int FLEX_PIN5 = A4;

int flexADC5 = 0;
int sensorMin5 = 1023;
int sensorMax5 = 0;
void setup()

{mySerial.begin(9600);
while (!Serial)
{
; // wait for serial port to connect. Needed for native USB poRT}
13
while(millis()<15000)
{if(digitalRead(7)==HIGH)
{
float flexADC1 = analogRead(FLEX_PIN1);
float flexADC2 = analogRead(FLEX_PIN2);
float flexADC3 = analogRead(FLEX_PIN3);
float flexADC4 = analogRead(FLEX_PIN4);
float flexADC5 = analogRead(FLEX_PIN5);

if(flexADC1<sensorMin1)
{sensorMin1=flexADC1;}

if(flexADC1>sensorMax1)
{sensorMax1=flexADC1;}

if(flexADC2<sensorMin2)
{sensorMin2=flexADC2;}

if(flexADC2>sensorMax2)
{sensorMax2=flexADC2;}

if(flexADC3<sensorMin3)
{sensorMin3=flexADC3;}

if(flexADC3>sensorMax3)
{sensorMax4=flexADC4;}

if(flexADC5<sensorMin5)
{sensorMin5=flexADC5;}

if(flexADC5>sensorMax5)
{sensorMax5=flexADC5;}

if(flexADC4<sensorMin4)
{sensorMin4=flexADC4;}

if(flexADC4>sensorMax4)
{sensorMax4=flexADC4;}

void printfun(char cp) //to avoid printing repeating symbols

{if(cp!=temp)
mySerial.print(cp);
temp=cp;}}
void loop()
{
// reading sensor value
float flexADC1 = analogRead(FLEX_PIN1);
float flexADC2 = analogRead(FLEX_PIN2);
float flexADC3 = analogRead(FLEX_PIN3);
float flexADC4 = analogRead(FLEX_PIN4);
float flexADC5 = analogRead(FLEX_PIN5);

flexADC1 = constrain(flexADC1,sensorMin1, sensorMax1);

flexADC2 = constrain(flexADC2,sensorMin2, sensorMax2);
flexADC3 = constrain(flexADC3,sensorMin3, sensorMax3);
flexADC4 = constrain(flexADC4,sensorMin4, sensorMax4);
flexADC5 = constrain(flexADC5,sensorMin5, sensorMax5);

float angle1= map(flexADC1, sensorMin1, sensorMax1, 0, 90);

14
float angle2= map(flexADC2, sensorMin2, sensorMax2, 0, 90);float angle3=
map(flexADC3, sensorMin3, sensorMax3, 0, 90);float angle4= map(flexADC4,
sensorMin4, sensorMax4, 0, 90);float angle5= map(flexADC5, sensorMin5,
sensorMax5, 0, 90);

xadc = analogRead(xpin);
yadc = analogRead(ypin);

if(((angle1>=70)&&(angle1<=82))&&((angle2>=77)&&(angle2<=95))&&((angle3>=70
)&&(angle3<=86))&&((angle4>=73)&&(angle4<=85))&&((angle5>=0)&&(angle5<=45))
)
printfun('A');
if(((angle1>=0)&&(angle1<=10))&&((angle2>=0)&&(angle2<=10))&&((angle3>=0)&&
(angle3<=12))&&((angle4>=0)&&(angle4<=10))&&((angle5>=65)&&(angle5<=80)))
printfun('B');
if(((angle1>=40)&&(angle1<=72))&&((angle2>=50)&&(angle2<=90))&&((angle3>=51
)&&(angle3<=75))&&((angle4>=42)&&(angle4<=66))&&((angle5>=34)&&(angle5<=50)
))
printfun('C');
if(((angle1>=50)&&(angle1<=72))&&((angle2>=45)&&(angle2<=90))&&((angle3>=35
)&&(angle3<=75))&&((angle4>=0)&&(angle4<=10))&&((angle5>=45)&&(angle5<=80))
&&!(((xadc>=412)&&(xadc<=418))&&((yadc>=340)&&(yadc<=360))))
printfun('D');
if(((angle1>=68)&&(angle1<=88))&&((angle2>=68)&&(angle2<=90))&&((angle3>=50
)&&(angle3<=80))&&((angle4>=54)&&(angle4<=80))&&((angle5>=58)&&(angle5<=88)
))
printfun('E');
if(((angle1>=0)&&(angle1<=10))&&((angle2>=0)&&(angle2<=10))&&((angle3>=0)&&
(angle3<=10))&&((angle4>=15)&&(angle4<=45))&&((angle5>=34)&&(angle5<=65)))
printfun('F');
if(((angle1>=75)&&(angle1<=90))&&((angle2>=75)&&(angle2<=90))&&((angle3>=65
)&&(angle3<=90))&&((angle4>=0)&&(angle4<=15))&&((angle5>=0)&&(angle5<=30))&
&(((xadc>=400)&&(xadc<=420))&&((yadc>=340)&&(yadc<=360))))
printfun('G');
if(((angle1>=70)&&(angle1<=85))&&((angle2>=75)&&(angle2<=90))&&((angle3>=0)
&&(angle3<=10))&&((angle4>=0)&&(angle4<=10))&&((angle5>=50)&&(angle5<=65))&
&!(((xadc>=410)&&(xadc<=420))&&((yadc>=368)&&(yadc<=380))))
printfun('H');
if(((angle1>=0)&&(angle1<=10))&&((angle2>=50)&&(angle2<=70))&&((angle3>=50)
&&(angle3<=70))&&((angle4>=50)&&(angle4<=70))&&((angle5>=50)&&(angle5<=85)&
&((xadc>=410)&&(xadc<=420))&&((yadc>=330)&&(yadc<=370))))
printfun('I');
if(((angle1>=0)&&(angle1<=10))&&((angle2>=50)&&(angle2<=70))&&((angle3>=50)
&&(angle3<=70))&&((angle4>=50)&&(angle4<=70))&&((angle5>=50)&&(angle5<=85))
&&(!((xadc>=410)&&(xadc<=420))&&((yadc>=355)&&(yadc<=370))))
printfun('J');
if(((angle1>=60)&&(angle1<=75))&&((angle2>=60)&&(angle2<=85))&&((a ngle3>=0)
&&(angle3<=10))&&((angle4>=0)&&(angle4<=15))&&((angle5>=30)&&(angle5<=55))&
&(((xadc>=404)&&(xadc<=415))&&((yadc>=368)&&(yadc<=380))))
printfun('K');
if(((angle1>=75)&&(angle1<=90))&&((angle2>=75)&&(angle2<=90))&&((angle3>=70
)&&(angle3<=90))&&((angle4>=0)&&(angle4<=15))&&((angle5>=0)&&(angle5<=30))&
&(((xadc>=390)&&(xadc<=405))&&((yadc>=360)&&(yadc<=380)))&&!((xadc>=270)&&(
xadc<=300))&&((yadc>=360)&&(yadc<=390)))
printfun('L');
if(((angle1>=40)&&(angle1<=61))&&((angle2>=72)&&(angle2<=84))&&((angle3>=45
)&&(angle3<=65))&&((angle4>=62)&&(angle4<=75))&&((angle5>=65)&&(angle5<=86)
))
15
printfun('M');
if(((angle1>=54)&&(angle1<=70))&&((angle2>=50)&&(angle2<=61))&&((angle3>=48
)&&(angle3<=66))&&((angle4>=60)&&(angle4<=76))&&((angle5>=50)&&(angle5<=65)
)&&(((xadc>=400)&&(xadc<=435))&&((yadc>=350)&&(yadc<=390))))
printfun('N');
if(((angle1>=68)&&(angle1<=88))&&((angle2>=68)&&(angle2<=90))&&((angle3>=50
)&&(angle3<=80))&&((angle4>=54)&&(angle4<=80))&&((angle5>=0)&&(angle5<=30))
)
printfun('O');
if(((angle1>=60)&&(angle1<=75))&&((angle2>=60)&&(angle2<=85))&&((angle3>=0)
&&(angle3<=10))&&((angle4>=0)&&(angle4<=15))&&((angle5>=30)&&(angle5<=55))&
&(((xadc>=270)&&(xadc<=290))&&((yadc>=360)&&(yadc<=380))))

printfun('P');
if(((angle1>=75)&&(angle1<=90))&&((angle2>=75)&&(angle2<=90))&&((angle3>=65
)&&(angle3<=90))&&((angle4>=0)&&(angle4<=15))&&((angle5>=0)&&(angle5<=30))&
&(((xadc>=270)&&(xadc<=300))&&((yadc>=360)&&(yadc<=390))))

printfun('Q');
if(((angle1>=40)&&(angle1<=72))&&((angle2>=45)&&(angle2<=90))&&((angle3>=20
)&&(angle3<=45))&&((angle4>=0)&&(angle4<=10))&&((angle5>=45)&&(angle5<=80))
&&(((xadc>=412)&&(xadc<=418))&&((yadc>=340)&&(yadc<=360))))

printfun('R');
if(((angle1>=70)&&(angle1<=90))&&((angle2>=80)&&(angle2<=90))&&((angle3>=80
)&&(angle3<=90))&&((angle4>=80)&&(angle4<=90))&&((angle5>=60)&&(angle5<=80)
))
printfun('S');
if(((angle1>=40)&&(angle1<=61))&&((angle2>=72)&&(angle2<=84))&&((angle3>=45
)&&(angle3<=65))&&((angle4>=44)&&(angle4<=63))&&((angle5>=65)&&(angle5<=86)
)&&(digitalRead(6)==HIGH))

printfun('T');
if(((angle1>=70)&&(angle1<=90))&&((angle2>=80)&&(angle2<=90))&&((angle3>=0)
&&(angle3<=10))&&((angle4>=0)&&(angle4<=10))&&((angle5>=60)&&(angle5<=80)))

printfun('U');
if(((angle1>=70)&&(angle1<=90))&&((angle2>=80)&&(angle2<=90))&&((angle3>=0)
&&(angle3<=10))&&((angle4>=0)&&(angle4<=10))&&((angle5>=60)&&(angle5<=80))&
&(digitalRead(6)==HIGH))

printfun('V');
if(((angle1>=70)&&(angle1<=90))&&((angle2>=0)&&(angle2<=10))&&((angle3>=0)&
&(angle3<=10))&&((angle4>=0)&&(angle4<=10))&&((angle5>=60)&&(angle5<=80)))

printfun('W');
if(((angle1>=50)&&(angle1<=72))&&((angle2>=45)&&(angle2<=90))&&((angle3>=35
)&&(angle3<=75))&&((angle4>=80)&&(angle4<=89))&&((angle5>=45)&&(angle5<=80)
))//&&!(((xadc>=412)&&(xadc<=418))&&((yadc>=340)&&(yadc<=360))))

printfun('X');
if(((angle1>=0)&&(angle1<=10))&&((angle2>=70)&&(angle2<=90))&&((angle3>=60)
&&(angle3<=80))&&((angle4>=80)&&(angle4<=90))&&((angle5>=15)&&(angle5<=35))
)
16
printfun('Y');

if(((angle1>=50)&&(angle1<=72))&&((angle2>=45)&&(angle2<=90))&&((angle3>=35
)&&(angle3<=75))&&((angle4>=0)&&(angle4<=10))&&((angle5>=45)&&(angle5<=80))
&&(((xadc>=412)&&(xadc<=418))&&((yadc>=340)&&(yadc<=360))))
printfun('Z');

delay(200);

17
Fig:6

18
 Fig:7

19
Humans possess the voice capability for interaction and communication among

each other. Unfortunately, not everybody has the capability of speaking and

hearing. Sign language used among the community of people who cannot speak

or hear as the means of communication. Sign language is a gesture representation

that involves simultaneously combining hand shapes, orientation and movement

of the hands and arms or body, and facial expressions to express fluently with a

speaker’s thoughts. The people who cannot speak makes use of the sign languages

to communicate with other fellow vocally impaired person and even with other

normal people who knows the meanings of sign languages or an interpreter is

needed to translate the meanings of sign languages to other people who can speak

and do not know the meanings of sign languages.

However, it is not always possible for an individual to be around all the time to

interpret the sign languages Hand Gesture Vocalizer for Dum band Deaf People

and not everybody can learn the sign languages. Thus, another alternative is that

we can use a computer or a smart phone as a mediator. The computer or a

smartphone could take an input from the vocally impaired person and give its

textual as well as and audio form of output.

20
This system is useful for deaf and dumb people to communicate withone another

and with common people. The dumb people use their sign language which is

difficult for common people and deaf peopleto understand. This system converts

the sign language into speech which is easy for deaf and normal people to

understand their language. The sign language is translated into some visual form,

to understand for the deaf people also. This text is display on LCD. Signlanguage

is a useful for communication between the deaf community and the normal

people. This project is basically designedto minimise the communication gap

between the dumb people andthe normal one. With this project the dumb people

can use the datagloves Which is used to perform sign language and it will be

converted into voice so that normal people can easily understand and also display

it on LCD so that people who cannot hear can read it on the screen.

21
Perfection in monitoring and sensing of the dynamic
movements involved in “Gesture Based Vocalizer”.

Designing of a jacket, which would be capable of recognizingthe gestures

and movements of animals.

Developing a custom vocalizer module that can provide morenumber of

voice outputs than the standard one available in the market while at the
same time cost less if considered for mass production.

Generate gestures for as many words as possible and developa machine

learning system that can interpret the various words generated and
develop a sentence on its own. This would reduce the load on the system
due to storage of large sentences and make the system more robust and
adaptive.

Interface with voice assistants like Siri or Alexa to generate voice outputs
rather than the vocalizer module. This would make the system more
compact and be used on various platforms like mobile phones that would
make it portable.

22
 Use hand gestures to control vehicles. This would beparticularly useful
for wheelchair bound people who find it difficult to manoeuvre it and
require assistance of someone.

Further integrated with various services and help togenerate employment for
the deaf and dumb people.

Geared up with the controller to provide home

automation on finger tips.

Paired up with fitness sensor to monitor health of theindividual.

23
1. Srinivas Gutta, Jeffrey Huang, Ibrahim F. Imam, and Harry Wechsler,“Face and
Hand Gesture Recognition Using Hybrid Classifiers”, ISBN: 0- 8186-7713-9/96,
pp.164-169

2. Jean-Christophe Lementec and Peter Bajcsy, “Recognition of Am Gestures

Using Multiple Orientation Sensors: Gesture Classification”, 2004 IEEE Intelligent
Transportation Systems Conference Washington, D.C., USA, October 34, 2004,
pp.965-970

3. Md. Al-Amin Bhuiyan, “On Gesture Recognition for Human-Robot Symbiosis”,

The 15th IEEE International Symposium on Robot and Human

4. Sanshzar Kettebekov, Mohammed Yeasin and Rajeev Sharma, “Improving

Continuous Gesture Recognition with Spoken Prosody”, Proceedings of the 2003 IEEE
Computer Society Conference on Computer Vision and Pattern Recognition
(CVPR’03), ISBN # 1063- 6919/03, pp.1-6

5. Masumi Ishikawa and Hiroko Matsumura,“Recognition of a Hand- Gesture

Based on Selforganization Using a Data Glove”, ISBN # 0-7803- 5871-6/99, pp. 739-
745

6. Prerana K.C, Nishu Mahato, and Bishruti Satyal, "Hand Gesture Recognition using
KINECT," Nepal Engineering College, Baktapur, Bachelor Thesis P3-I.0-14-4,
2015.

24
7. M. S Kasar, Anvita Deshmukh, Akshada Gavande, and PriyankaGhadage,
"Smart Speaking GloveVirtual tongue for Deaf and Dumb,"Internaltional Journal of
Advanced Research in Electrical, Electronicsand Instrumentation Engineering, vol.
5, no. 3, p. 7, Mar. 2016.

8. Savan Patel. (2017, May) medium.com. [Online].

25