Hand Gesture Control for Automated Digital

Presentations

Vijaya Prakash R1[0000-0003-2177-5350], Maduri Ram Charan Teja2, Suhaas Sanga3,

Renukuntla Dhanush4, Kothapally Prem5, Gurrapu Aditya Krishna6
1,2,3,4,5,6
School of Computer Science & Artificial Intelligence, SR University, Warangal, India
1
[email protected], [email protected],
3
[email protected], [email protected],
5
[email protected], [email protected]

Abstract. This project integrates computer vision and gesture recognition tech-
niques to develop an interactive slideshow navigation system. The program
utilizes the OpenCV library for image processing and the CV zone library for
hand tracking. Users can control the slideshow by performing specific hand
gestures in front of a webcam. The system begins by allowing the user to select
a folder containing PNG images, which are then sequentially renamed. The
main functionality involves gesture-based control for navigating through the
images in the slideshow. Hand gestures, detected using the Hand Tracking
Module, are mapped to actions such as moving to the previous or next slide,
erasing annotations, and showing and drawing pointers on the images. Addi-
tionally, the system provides real-time feedback by displaying the webcam feed
alongside the slideshow. The interactive nature of this project makes it suitable
for presentations or educational purposes where users can dynamically interact
with the displayed content.

Keywords: Recognition, Machine Learning, Gesture Detection, Gesture Classi-

fication

1 Introduction

In an era characterized by digital communication and information dissemination, the

art of effective presentation holds a crucial place [1]. Traditional means of navigating
presentation slides, involving peripheral devices like mice, keyboards, or laser point-
ers, often demand prior technical knowledge, creating barriers for users [2]. However,
this project introduces an innovative solution that leverages the power of dynamic
hand gestures and OpenCV technology to redefine the way we interact with presenta-
tions. The core of this endeavor is the use of a camera to capture and interpret six
distinct hand gestures. Each of these gestures triggers specific actions within the pre-
sentation, enabling presenters to seamlessly navigate between slides, write or erase
content, highlight sections, and even terminate the presentation, all through intuitive
hand movements [3]. What sets this system apart is its ability to accomplish all this
without the need for additional hardware, specialized gloves, or markers, making it a
2

cost-effective and user-friendly alternative. The underlying technology is built upon

the robust foundation of the Python framework, with critical components including
OpenCV, CV Zone, NumPy, and Media Pipe [4][5].
Through the fusion of machine learning and motion image-based techniques, the
system accurately recognizes intricate hand motions, empowering presenters to com-
municate nonverbally, engage their audiences, and maintain precise control over their
presentations. This project represents a groundbreaking fusion of machine learning
and computer vision to create a versatile human-machine interface that enhances the
traditional presentation experience. By using gestures such as swiping, giving a
thumbs-up, or halting, users can effortlessly command their presentation slides, sig-
nificantly improving the fluidity and expressiveness of their presentations. Ultimately,
the project's goal is to empower presenters by providing a more natural and interactive
means of controlling presentations, thereby enhancing the overall impact and effec-
tiveness of their messages. In the digital age, this dynamic hand gesture-based control
system promises to revolutionize the art of presentations, offering a modern and en-
gaging tool for communicators.

2 Literature Survey

In their study, authors Devivara Prasad et. al. [6], explores the significance of gesture
recognition in Human-Computer Interaction (HCI), emphasizing its practical
applications for individuals with hearing impairments and stroke patients. They used
image feature extraction tools and AI-based classifiers for 2D and 3D gesture
recognition. Their proposed system harnesses machine learning, and real-time image
processing with Media Pipe, and OpenCV to enable efficient and intuitive
presentation control using hand gestures, addressing the challenges of accuracy and
robustness. The research focuses on enhancing the user experience, particularly in
scenarios where traditional input devices are impractical, highlighting the potential of
gesture recognition in HCI.[13][15]
Reethika et. al. [7], presents a study on Human-Computer Interaction (HCI) with a
focus on hand gesture recognition as a natural interaction technique. It explores the
significance of real-time hand gesture recognition, particularly in scenarios where
traditional input devices are impractical. The methodology involves vision-based
techniques that utilize cameras to capture and process hand motions, offering the
potential to replace conventional input methods. The paper discusses the advantages
and challenges of this approach, such as the computational intensity of image
processing and privacy concerns regarding camera usage. Additionally, it highlights
the benefits of gesture recognition for applications ranging from controlling computer
mouse actions to creating a virtual HCI device [16].
Hajeera Khanum [8], outlines a methodology that harnesses OpenCV and Google's
MediaPipe framework [167[18] to create a presentation control system that interprets
hand gestures. Using a webcam, the system captures and translates hand movements
into actions such as slide control, drawing on slides, and erasing content, eliminating
the need for traditional input devices. While the paper does not explicitly enumerate
 3

the challenges encountered during system development, common obstacles in this

field may include achieving precise gesture recognition, adapting to varying lighting
conditions, and ensuring the system's reliability in real-world usage scenarios. This
work contributes to the advancement of human-computer interaction, offering a
modern and intuitive approach to controlling presentations through hand gestures.[19]
Salonee et. al. [9] introduces a system that utilizes artificial intelligence-based hand
gesture detection, employing OpenCV and MediaPipe. The system allows users to
control presentation slides via intuitive hand gestures, eliminating the reliance on
conventional input devices like keyboards or mice. The gestures correspond to various
actions, including initiating presentations, pausing videos, transitioning between
slides, and adjusting volume. This innovative approach enhances the natural
interaction between presenters and computers during presentations, demonstrating its
potential in educational and corporate settings. Notably, the paper does not explicitly
detail the challenges encountered during the system's development, but it makes a
valuable contribution to the realm of human-computer interaction by rendering digital
presentations more interactive and user-friendly. [20]
Bobo Zeng et. al [10], present a real-time interactive presentation system that
utilizes hand gestures for control. The system integrates a thermal camera for robust
human body segmentation, overcoming issues with complex backgrounds and varying
illumination from projectors. They propose a fast and robust hand localization
algorithm and a dual-step calibration method for mapping interaction regions between
the thermal camera and projected content using a web camera. The system has high
recognition rates for hand gestures, enhancing the presentation experience. However,
the challenges they encountered during development, such as the need for precise
calibration and handling hand localization, are not explicitly mentioned in the paper.
[21]
In Meera Paulson et. al. [11] introduces a gesture recognition system for enhancing
presentations and enabling remote control of electronic devices through hand
gestures. It incorporates ATMEGA 328, Python, Arduino, Gesture Recognition,
Zigbee, and wireless transmission [22]. The paper emphasizes the significance of
gesture recognition in human-computer interaction, its applicability in various
domains, and its flexibility to cater to diverse user needs. The system offers features
such as presentation control, home automation, background change, and sign
language interpretation. The authors demonstrated a cost-effective prototype with
easy installation and extensive wireless signal transmission capabilities. The paper
discusses the results, applications, methodology, and challenges, highlighting its
potential to improve human-machine interaction across different fields.
Rina Damdoo et. al. [12], present a vision-based adaptive hand gesture recognition
system employing Convolutional Neural Networks (CNN) for machine learning
classification. The study addresses the challenges of recognizing dynamic hand
gestures in real-time and focuses on the impact of lighting conditions. The authors
highlight that the performance of the system significantly depends on lighting
conditions, with better results achieved under good lighting. They acknowledge that
developing a robust system for real-time dynamic hand gesture recognition,
particularly under varying lighting conditions, is a complex task. The paper offers
4

insights into the potential for further improvement and the use of filtering methods to
mitigate the effects of poor lighting, contributing to the field of dynamic hand gesture
recognition.
RutikaBhor et. al. [13] presents a real-time hand gesture recognition system for
efficient human-computer interaction. It allows remote control of PowerPoint
presentations through simple gestures, using Histograms of Oriented Gradients and K-
Nearest Neighbor classification with around 80% accuracy. The technology extends
beyond PowerPoint to potentially control various real-time applications. The paper
addresses challenges in creating a reliable gesture recognition system and optimizing
lighting conditions. It hints at broader applications, such as media control, without
intermediary devices, making it relevant to the human-computer interaction field.
References cover related topics like gesture recognition in diverse domains.

3 Methodology

The project's primary objective is to make the presentation easy for the presenter to
deliver in a comfortable by controlling the complete presentation through hand
gestures.

Fig.1 Cyclic Process

The whole concept of this project is demonstrated in the Fig.1 It gives a complete
step by step process from uploading of files to till terminated of the presentation.
3.1 Data Collection

In this project the input data is given by the user in the form of ppt slides in images
format where the user will convert the ppt slides into images and those images will be
stored in a folder. The folder with images is the data for this project, specified in the
Fig. 1.
 5

3.2 Data Preprocessing

To rename and organize a set of PNG images, the initial step involves assigning
sequential numbers to them in the desired order. This can be achieved through
scripting or batch operations using programming or command-line tools. Once
renamed, the images will have consecutive identifiers, making it easier to organize
and retrieve them in a logical order.
After successfully renaming the PNG images with sequence numbers, the next step
is to sort them based on these assigned numerical values. Sorting ensures that the
images are used in the correct order, following the numerical sequence. This process
is crucial when creating presentations (PPT) or when a specific order is required for
image usage, as it ensures that the images are in the desired sequence for easy access
and presentation purposes. Overall, these procedures simplify the task of organizing
and working with PNG images in a structured and orderly manner. After uploading the
files folder, the data preprocessing starts renaming the images and sorting immediately and
storing them back in the folder takes place as show in Fig.2.

Fig.2 Preprocessing of Data

Hand Detection: The method recognizes and localizes a hand's position within a
video frame. The hand detection is the key objective in this research, and we em -
ployed the Kanade-Lucas-Tomasi (KLT) algorithm to identify and locate all known
objects in a scene [14]. The algorithm starts by identifying feature points in the first
frame of a video or image sequence. These features could include corners, edges, or
any other distinguishing points in the image. The Harris corner detector [15] is com-
monly used for feature detection. It detects corners by analyzing intensity changes in
various directions. Once the features are identified in the first frame, the algorithm
attempts to track them in subsequent frames. It is assumed that the features move in
small steps between frames.
A small window is considered around each initial frame feature point. The
algorithm searches the next frame for the best window match. Feature point optical
flow is estimated using the Lucas-Kanade method [10]. The motion is assumed to be
6

constant in a local neighborhood around the feature point. The optical flow equation
is solved for each window pixel around the feature point. Motion parameters (w) and
spatial intensity gradients (Ix and Iy) are related by this equation. The KLT algorithm
analyzes spatial gradient matrix as specified in equation 1, eigenvalues to determine
feature point tracking reliability. Spatial gradients of intensity in the window around
the feature point determine the matrix. A feature point is reliable for tracking if its
matrix eigenvalues are above a threshold. Fig. 3 describes the tracking of the hand
with the help of matrix eigen values.

(1)
[ ∑ w2 I 2x ∑ w2 I x I y
∑ w 2 I x I y ∑ w 2 I 2y ]
Finger Tracking: After detecting the hand, the algorithm records the location of
individual fingers. It may entail estimating hand landmarks to pinpoint crucial spots
on the fingers, particularly the fingertips.

Finger State Classification: The algorithm defines each finger's state as "up" (1) or
"down" (0) based on its location and movement. To establish these classifications, it
most likely evaluates the angles and placements of the fingers compared to a refer-
ence hand form.

Finger State Combination: The algorithm creates a combination of finger states for
the entire hand. For instance, if all fingers are labeled "up," it may indicate "5". If all
the fingers are marked "down," it may indicate "0."

Fig.4. Hand Track Mechanism

4 Results

The hand tracking mechanism, finger state classification, and combination allow each
finger to be identified and assigned to a specific task. Figure 4 depicts this classifica-
tion for the purpose of presentation. The first gesture is used to move the slide to the
previous slide, the second gesture is used for the next slide, the third one is used for
 7

the pointer to point the object on the slide, the fourth gesture is used to delete the
object drawn with the help of the fifth gesture, and the final gesture is used to exit the
presentation.

Fig. 4. Gestures to control the Presentation.

There were several experiments that we carried out in order to assess the effectiveness
of the system. The first experiment was designed to determine how accurate the de-
tection and classification of hand gestures turned out to be. We discovered that the
system was able to accurately detect and categorize hand gestures in most situations.
Figure 5 Shows the hand tracking and gesture accuracy with the help of KLT algo-
rithm of the system, an accuracy rate of approximately 95%. We conducted a second
experiment in which we examined the system's capability of controlling a presentation
with hand gestures. It was discovered by us that the system was able to control the
slides in a smooth manner and carry out a variety of actions, such as moving forward
or going back to the slide that came before it.

Fig. 5. Gestures to control the Presentation.

In the current model, we simply set the gesture array using the built-in Hand
Tracking Module, saving time on training, and collecting hand gestures. Converting
PowerPoint to images and uploading them will take very little time. The accuracy of
the built-in model ranges from 95 to 97%. The previous model required more time for
8

hand tracking because there was no built-in model for detecting hand gestures, and
the accuracy was less than 95%.
For this project HD camera is mandatory, the range of normal inbuilt cameras in
existing laptops is 5 meters. To get a long range of gesture recognition we need to use
external long-range cameras. Once the termination gesture is used the files will be
deleted. If the user wants to use the files again then they should upload the files again.

5 Conclusion

This project is an innovative and interactive presentation control system that utilizes
computer vision and gesture recognition. It offers a hands-free and engaging way to
interact with your presentation slides. With the ability to control slide navigation
through specific hand gestures, such as moving to the next or previous slide, your
project provides a convenient and intuitive alternative to traditional clickers or
keyboard shortcuts. Additionally, the option to write on the slides and erase content
by making hand movements enhances the interactivity of your presentations. The
pointer highlighter feature allows you to draw attention to specific details on your
slides, making it a powerful tool for emphasizing key points.
Furthermore, the capability to terminate the presentation with a gesture provides an
efficient way to conclude your talk. Overall, your project empowers presenters to
connect with their audience more dynamically and engagingly, all while using the
OpenCV interface and hand-tracking technology. It is a valuable addition to the realm
of presentation tools, enabling more interactive and captivating communication. The
future enhancement for this will be adding voice commands along with the hand
gesture and presenter recognition.
The challenges in this are that we are currently using 6 gestures to control the pre-
sentation, but we can improve it by adding more gestures. We intend to add speech
commands to perform operations such as moving the slides back and forth.

References
1. D.O. Lawrence, and M.J. Ashleigh, Impact Of Human-Computer Interaction (HCI) on
Users in Higher Educational System: Southampton University As A Case Study, Vol.6, No
3, pp. 1-12, September (2019)
2. Sebastian Raschka, Joshua Patterson, and Corey Nolet, Machine Learning in Python: Main
Developments and Technology Trends in Data Science, Machine Learning, and Artificial
Intelligence, (2020)
3. Xuesong Zhai, Xiaoyan Chu, Ching Sing Chai, Morris Siu Yung Jong, Andreja Istenic,
Michael Spector, Jia-Bao Liu, Jing Yuan, Yan Li, A Review of Articial Intelligence (AI) in
Education from 2010 to 2020, (2021)
4. D. Jadhav, Prof. L.M.R.J. Lobo, Hand Gesture Recognition System to Control Slide Show
Navigation IJAIEM, Vol. 3, No. 4 (2014)
5. Ren, Zhou, et al. Robust part-based hand gesture recognition using Kinect sensor, IEEE
Transactions on multimedia 15.5, pp.1110-1120, (2013), Page 8-11.
 9

6. Devivara Prasad G, Mr. Srinivasulu M. "Hand Gesture Presentation by Using Machine

Learning." September 2022, IJIRT, Volume 9, Issue 4.
7. G. Reethika, P.Anuhya, M. Bhargavi. "Slide Presentation by Hand Gesture Recognition
Using Machine Learning", IRJET, Volume 10, Issue: 01, Jan 2023
8. Hajeera Khanum, Dr. Pramod H B. "Smart Presentation Control by Hand Gestures Using
Computer Vision and Google’s Mediapipe", IRJET, Volume: 09 Issue: 07, July 2022.
9. Salonee Powar, Shweta Kadam, Sonali Malage, Priyanka Shingane. "Automated Digital
Presentation Control using Hand Gesture Technique.", ITM Web of Conferences 44,
03031 (2022).
10. G. L. P, A. P, G. Vinayan, G. G, P. M and A. S. H, "Lucas Kanade based Optical Flow for
Vehicle Motion Tracking and Velocity Estimation," 2023 International Conference on
Control, Communication and Computing (ICCC), Thiruvananthapuram, India, 2023, pp. 1-
6, doi: 10.1109/ICCC57789.2023.10165227.
11. Meera Paulson, Nathasha P R, Silpa Davis, Soumya Varma, "Smart Presentation Using
Gesture Recognition", 2017, Volume 2, Issue 3.
12. Rina Damdoo, Kanak Kalyani, Jignyasa Sanghavi, "Adaptive Hand Gesture Recognition
System Using Machine Learning Approach", Biosc Biotech Res Comm. Special Issue Vol
13 No 14 (2020) pp.106-110.
13. Bhor Rutika, Chaskar Shweta, Date Shraddha, Auti M. A, “PowerPoint Presentation Con-
trol Using Hand Gestures Recognition”, International Journal of Research Publication and
Reviews, Vol 4, no 5, pp 5865-5869, May 2023
14. D. Mikhaylov, A. Samoylov, P. Minin and A. Egorov, "Face Detection and Tracking from
Image and Statistics Gathering," 2014 Tenth International Conference on Signal-Image
Technology and Internet-Based Systems, Marrakech, Morocco, 2014, pp. 37-42, doi:
10.1109/SITIS.2014.85.
15. Zhao, J.; Su, L.; Wang, X.; Li, J.; Yang, F.; Jiang, N.; Hu, Q. DTFS-eHarris: A High Ac -
curacy Asynchronous Corner Detector for Event Cameras in Complex Scenes. Appl. Sci.
2023, 13, 5761. https://doi.org/10.3390/app13095761
16. M. F. Wahid, R. Tafreshi, M. Al-Sowaidi and R. Langari, "An efficient approach to recog-
nize hand gestures using machine-learning algorithms," 2018 IEEE 4th Middle East Con-
ference on Biomedical Engineering (MECBME), Tunis, Tunisia, 2018, pp. 171-176, doi:
10.1109/MECBME.2018.8402428.
17. Ajay Talele, Aseem Patil, Bhushan Barse, “Detection of Real-Time Objects Using Tensor-
Flow and OpenCV”, Asian Journal of Convergence in Technology, Vol 5, (2019)
18. Ahmed Kadem Hamed AlSaedi, Abbas H. Hassin Al Asadi, “A New Hand Gestures
Recognition System”, Indonesian Journal of Electrical Engineering and Computer Science,
Vol 18, (2020)
19. Sebastian Raschka, Joshua Patterson, and Corey Nolet, “Machine Learning in Python:
Main Developments and Technology”, Trends in Data Science, Machine Learning, and
Artificial Intelligence, (2020)
20. I. Dhall, S. Vashisth, G. Aggarwal, Automated Hand Gesture Recognition using a Deep
Convolutional Neural Network, 10th International Conference on Cloud Computing, Data
Science & Engineering, (2020)
21. Meera Paulson, Natasha, Shilpa Davis on the Smart presentation using gesture recognition
and OpenCV, Asian Journal of Convergence in Technology, Vol 5, (2019)