International Journal of Scientific and Research Publications, Volume 4, Issue 12, December 2014 1

ISSN 2250-3153

Hand Gesture Recognition Techniques For Human

Computer Interaction Using OpenCv
Sajjad Ur Rahman*, Zeenat Afroze**, Mohammed Tareq***
*
Dept. Of EEE, American International University-Bangladesh (AIUB), Banani, Dhake-1213
**
Dept. Of EEE, American International University-Bangladesh (AIUB), Banani, Dhake-1213
* **
Dept. Of EETE, Dhaka International University, Green Road, Dhake-1205

Abstract- A few decades ago, usage of computer was so tough and complicated that none but scientists could use it. But now with
the passage of time and to cope with the demand of our every sphere of life, usage of computer has become so easy that everyone can
use it. At present, mouse or keyboard is being used to interact with the computer. But sometimes it seems uncomfortable because
people don’t want to get off from where they are sitting or lying. This paper proposed a technique for man-machine interaction which
is based on gesture recognition using openCV technology which provides basic data structures for image processing with most
favorable efficiency [1]. A hand image is taken as input to detect the hand easily in this system. In this paper, all the hand gesture
images are captured from a single web camera. The proposed algorithm also helps to locate the palm and fingertip in hand gesture.

Index Terms- Human Computer Interaction, Hand Gesture, Contour, Convex-Hull, Convexity Defects.

I. INTRODUCTION

H uman Computer Interaction (HCI) is one of the most outstanding inventions which works to find more effective method to
develop the current system. Among them, the system of using a mouse or a keyboard is the most popular. But now a new
technology has been arrived for Human Computer Interaction (HCI) which is based on gesture recognition means a move to convey
intention or to evoke response. Actually gesture technique is used who can understand it and is being used today to control the
computer instead of mouse or keyboard. This paper presents three new interactive methods which will make the system more reliable
[2][9][10]. The paper also describes an effective hand based interaction method that needs preprocessing means conversion of the
user’s hand image from one form to another form for implementation that makes a connection with the next step which leads to hand
detection. As many new technologies have been arrived to do our work and among them hand gesture based technology is the latest,
so its usage is very limited. This technique is being used in television, computer, and robots without using remote control, joystick to
complete the least job [5][6]. It will also be used in some applications such as holographic technology, gadgets like phone, editing and
copying data, creating programs etc. Section II contained the method and steps of how hand gesture process works. Section III
contained the simulation part of hand detection and described the result. Section IV included the future work of hand gesture
recognition and section V contained the conclusion part of this paper.

II. METHODOLOGY
The whole systems are based on contour of hand. Contours are the line or surface of which no part is straight or flat connecting all
the continuous points throughout the boundary. This process is executed after thresholding. A simple hand image is taken which is in
RGB format captured by a web camera. Some preliminary process need to be performed to generate a binary image which provides
information of hand contour [3]. Here binary image is used to determine the contour and convex-hull of contour. The convex-hull of a
set of points is the smallest outline of a closed figure encircling the points. It is drawn on all sides of contour of hand as if every
contour point is in the limits of convex-hull. It creates a folded paper container around contour of hand. A defect is seen in the hand
gesture when convex-hull is drawn on every side of contour of hand which fits sets of contour points of the hand. It uses maximum
positions to construct the hull to assemble every contour point inside. Convexity defects give set of values in the vector form [7][8].
The fingertip position is detected from contour of hand and palm position is determined by the information which is extracted from
convex-hull [9]. As computation geometry algorithm performed, so computation cost is low in this system.

www.ijsrp.org
International Journal of Scientific and Research Publications, Volume 4, Issue 12, December 2014 2
ISSN 2250-3153

Binary Image

Computation Geometry
Input Image Preliminary Processing
Calculation

Contour, Convex-Hull and

Fingertips and Palm Detection
Convexity Defects

Figure 1: The flow diagram of implemented algorithm

III. SIMULATIONS AND RESULTS

A. Hand Segmentation
Hand segmentation is an important step before hand detection which is used to remove the hand image from background portion. This
process is accomplished by using preliminary process which is transformation and thresholding. In this algorithm, at first RGB image
taken as input from a web camera. Then RGB image is converted into YCbCr image. At last output of thresholding can be found from
converting YCbCr image to binary image that consists of 0 and 1. So where there is hand image, ‘1’ will be putted and where there is
background, ‘0’ will be putted there [10].

Figure 2: Input Image Frame and YCbCr Image

Figure 3: Threshold Image

www.ijsrp.org
International Journal of Scientific and Research Publications, Volume 4, Issue 12, December 2014 3
ISSN 2250-3153

In Figure 3, hand color turns into white and background color turns into black. This type of image is easy to detect the shape of
boundary.

B. Hand Detection

Contours
A hand image and its shape of user is detected and recognized by using contours analysis [4]. In the hand image, a contour is a
series of points which are the boundary pixels of an area that is shown in Figure 4.

Figure 4: Detected Contour of Input Image

Convex-Hull
Figure 5 shows an example of convex-hull that connects lines of continuous points which never exceed the contour of hand. The
heptagon in the figure is the convex-hull of set. These seven points that forms the heptagon are known as hull points.

Figure 5: Convex-Hull of Input Image

Convexity Defects

Figure 6 illustrates the convexity defects of a hand image. The grey lines on the contour indicate the convex-hull of the hand. The
regions in ash color are contained in the convex-hull. But they are not contained in the hand image. These regions are so called
convexity defects and the yellow colored points shows the defect [6].

www.ijsrp.org
International Journal of Scientific and Research Publications, Volume 4, Issue 12, December 2014 4
ISSN 2250-3153

Figure 6: Convexity Defects of Input Image

Fingertip and Palm Area Detection

In this method of interaction [9], fingertip and palm area of hand image is determined by making the use of defect points present in
the hand gesture. In the Figure 7, green color indicates fingertip, yellow color indicates defects point and blue circle shows palm
position.

Showing 0 Showing 1

Showing 2 Showing 3

www.ijsrp.org
International Journal of Scientific and Research Publications, Volume 4, Issue 12, December 2014 5
ISSN 2250-3153

Showing 4 Showing 5

Figure 7: Detected Fingertip and Palm of the hand

IV. FUTURE WORK

The system or technology, that we use now, still neither flawless nor applicable in every required situation properly, though it is the
latest technology. Future development or research in the sector determines to remove all these flaws so that it can be more efficient
and reliable.

V. CONCLUSION

In this system, different effective techniques for man-machine interaction are observed. A few systems for preprocessing of input
image are presented. This paper also introduced about fingertip and palm detection in the hand gesture which increases the freedom of
usability. These methods are going to use in different applications in future.

ACKNOWLEDGMENT
I would like to thank Dr. Tanmoy Debnath, assistant professor of department of EEE for his guidance during work on the
implementation of these techniques and while writing this paper.

REFERENCES

[1] OpenCV usage documentationhttp://docs.opencv.org

[2] Meenakshi Panwar and Pawan Singh Mehra , “Hand Gesture Recognition for Human Computer Interaction”, in Proceedings of IEEE International Conference on
Image Information Processing(ICIIP 2011), Waknaghat, India, November 2011
[3] Amiraj Dhawan, Vipul Honrao”implementation of hand detection based techniques for human computer interaction”, in International Journal of Computer
Applications (0975 – 8887)
[4] Contours, Convex Hull and Convexity Defects http://docs.opencv.org/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html
[5] www.google.com
[6] www.ieeeexplore.com
[7] W. Z. Chen, “Real-time palm tracking and hand gesture estimation based on fore-arm contour,” M. S. Thesis, Dept. Inform. Eng.,National Taiwan University of
Science and Technology, 2011.
[8] Kim, S., S. Takahashi and J. Tanaka, 2010. New interface using palm and fingertip without marker for ubiquitous environment. Proceedings of the 9th
International Conference on Computer and Information Science, Aug. 18-20, IEEE Xplore Press, Yamagata, pp: 819-824. DOI: 10.1109/ICIS.2010.110
[9] Meenakshi Panwar, “Hand Gesture Recognition based on Shape Parameters”.
[10] Thresholdinghttp://www.cse.unr.edu/~bebis/CS791E/Notes/Thresholding.pdf

www.ijsrp.org
International Journal of Scientific and Research Publications, Volume 4, Issue 12, December 2014 6
ISSN 2250-3153

AUTHORS
First Author – Sajjad Ur Rahman is a student of Dept. of MEEE of American International University Bangladesh. He obtained
B.Sc. in Electrical and Electronic Engineering (EEE) from American International University-Bangladesh (AIUB) in 2013.

Second Author – Zeenat Afroze was born in Dhaka, Bangladesh on 1981. She completed B.Sc. on Electrical and Electronic
Engineering from Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh in 2004. She also completed
M.Sc. on Communication Engineering and Media Technology from Universität Stuttgart, Stuttgart, Germany in 2008. Currently, she
is working as an Assistant professor in American International University Bangladesh. Her research interest includes Communication
and Digital Signal Processing.

Third Author – Mohammad Tareq is serving as Asst. Prof. at the dept. of EETE of Dhaka International University (DIU), Dhaka
since June 2014. He worked as lecturer at the dept. of EETE of same university from March 2011 to May 2014. He obtained B.Sc. in
Electrical and Electronic Engineering (EEE) from American International University-Bangladesh (AIUB) in 2009. At present he is
pursuing M.Sc. in Electrical and Electronic Engineering (MEEE) at AIUB. His fields of interests are antenna design, mobile
communications, optoelectronics and renewable energy.

www.ijsrp.org