Chapter 26

Hand Gesture Control for Automated

Digital Presentations

Vijaya Prakash Rajanala , Maduri Ram Charan Teja, Suhaas Sanga,

Renukuntla Dhanush, Kothapally Prem, and Gurrapu Aditya Krishna

Abstract This project integrates computer vision and gesture recognition techniques
to develop an interactive slideshow navigation system. The program utilizes the
OpenCV library for image processing and the CV zone library for hand tracking.
Users can control the slideshow by performing specific hand gestures in front of
a webcam. The system begins by allowing the user to select a folder containing
PNG images, which are then sequentially renamed. The main functionality involves
gesture-based control for navigating through the images in the slideshow. Hand
gestures, detected using the Hand Tracking Module, are mapped to actions such as
moving to the previous or next slide, erasing annotations, and showing and drawing
pointers on the images. Additionally, the system provides real-time feedback by
displaying the webcam feed alongside the slideshow. The interactive nature of this
project makes it suitable for presentations or educational purposes where users can
dynamically interact with the displayed content.

Keywords Recognition · Machine learning · Gesture detection · Gesture

classification

26.1 Introduction

In an era characterized by digital communication and information dissemination,

the art of effective presentation holds a crucial place [1]. Traditional means of navi-
gating presentation slides, involving peripheral devices like mice, keyboards, or laser
pointers, often demand prior technical knowledge, creating barriers for users [2].
However, this project introduces an innovative solution that leverages the power of
dynamic hand gestures and OpenCV technology to redefine the way we interact

V. P. Rajanala · M. R. C. Teja (B) · S. Sanga · R. Dhanush · K. Prem · G. A. Krishna

School of Computer Science and Artificial Intelligence, SR University, Warangal, India
e-mail: [email protected]
V. P. Rajanala
e-mail: [email protected]
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 327
P. K. Jha et al. (eds.), Proceedings of the Second Congress on Control, Robotics,
and Mechatronics, Smart Innovation, Systems and Technologies 409,
https://doi.org/10.1007/978-981-97-7094-6_26
328 V. P. Rajanala et al.

with presentations. The core of this endeavor is the use of a camera to capture and
interpret six distinct hand gestures. Each of these gestures triggers specific actions
within the presentation, enabling presenters to seamlessly navigate between slides,
write or erase content, highlight sections, and even terminate the presentation, all
through intuitive hand movements [3]. What sets this system apart is its ability to
accomplish all this without the need for additional hardware, specialized gloves,
or markers, making it a cost-effective and user-friendly alternative. The underlying
technology is built upon the robust foundation of the Python framework, with critical
components including OpenCV, CV Zone, NumPy, and Media Pipe [4, 5].
Through the fusion of machine learning and motion image-based techniques,
the system accurately recognizes intricate hand motions, empowering presenters
to communicate nonverbally, engage their audiences, and maintain precise control
over their presentations. This project represents a groundbreaking fusion of machine
learning and computer vision to create a versatile human–machine interface that
enhances the traditional presentation experience. By using gestures such as swiping,
giving a thumbs-up, or halting, users can effortlessly command their presentation
slides, significantly improving the fluidity and expressiveness of their presenta-
tions. Ultimately, the project’s goal is to empower presenters by providing a more
natural and interactive means of controlling presentations, thereby enhancing the
overall impact and effectiveness of their messages. In the digital age, this dynamic
hand gesture-based control system promises to revolutionize the art of presentations,
offering a modern and engaging tool for communicators.

26.2 Literature Survey

In their study, authors Devivara Prasad et al. [6] explored the significance of gesture
recognition in Human–Computer Interaction (HCI), emphasizing its practical appli-
cations for individuals with hearing impairments and stroke patients. They used
image feature extraction tools and AI-based classifiers for 2D and 3D gesture recog-
nition. Their proposed system harnesses machine learning, and real-time image
processing with Media Pipe, and OpenCV to enable efficient and intuitive presenta-
tion control using hand gestures, addressing the challenges of accuracy and robust-
ness. The research focuses on enhancing the user experience, particularly in scenarios
where traditional input devices are impractical, highlighting the potential of gesture
recognition in HCI [13, 15].
Reethika et al. [7] presented a study on Human–Computer Interaction (HCI) with
a focus on hand gesture recognition as a natural interaction technique. It explores the
significance of real-time hand gesture recognition, particularly in scenarios where
traditional input devices are impractical. The methodology involves vision-based
techniques that utilize cameras to capture and process hand motions, offering the
potential to replace conventional input methods. The paper discusses the advan-
tages and challenges of this approach, such as the computational intensity of image
processing and privacy concerns regarding camera usage. Additionally, it highlights
26 Hand Gesture Control for Automated Digital Presentations 329

the benefits of gesture recognition for applications ranging from controlling computer
mouse actions to creating a virtual HCI device [16].
Khanum and Pramod [8] outlined a methodology that harnesses OpenCV and
Google’s MediaPipe framework [16–18] to create a presentation control system that
interprets hand gestures. Using a webcam, the system captures and translates hand
movements into actions such as slide control, drawing on slides, and erasing content,
eliminating the need for traditional input devices. While the paper does not explic-
itly enumerate the challenges encountered during system development, common
obstacles in this field may include achieving precise gesture recognition, adapting to
varying lighting conditions, and ensuring the system’s reliability in real-world usage
scenarios. This work contributes to the advancement of human–computer interac-
tion, offering a modern and intuitive approach to controlling presentations through
hand gestures [19].
Powar et al. [9] introduced a system that utilizes artificial intelligence-based hand
gesture detection, employing OpenCV and MediaPipe. The system allows users
to control presentation slides via intuitive hand gestures, eliminating the reliance
on conventional input devices like keyboards or mice. The gestures correspond
to various actions, including initiating presentations, pausing videos, transitioning
between slides, and adjusting volume. This innovative approach enhances the natural
interaction between presenters and computers during presentations, demonstrating
its potential in educational and corporate settings. Notably, the paper does not explic-
itly detail the challenges encountered during the system’s development, but it makes
a valuable contribution to the realm of human–computer interaction by rendering
digital presentations more interactive and user-friendly [20].
Gokul et al. [10] presented a real-time interactive presentation system that utilizes
hand gestures for control. The system integrates a thermal camera for robust human
body segmentation, overcoming issues with complex backgrounds and varying illu-
mination from projectors. They proposed a fast and robust hand localization algo-
rithm and a dual-step calibration method for mapping interaction regions between
the thermal camera and projected content using a web camera. The system has high
recognition rates for hand gestures, enhancing the presentation experience. However,
the challenges they encountered during development, such as the need for precise
calibration and handling hand localization, are not explicitly mentioned in the paper
[21].
Paulson et al. [11] introduced a gesture recognition system for enhancing presen-
tations and enabling remote control of electronic devices through hand gestures.
It incorporates ATMEGA 328, Python, Arduino, Gesture Recognition, Zigbee, and
wireless transmission [22]. The paper emphasizes the significance of gesture recog-
nition in human–computer interaction, its applicability in various domains, and its
flexibility to cater to diverse user needs. The system offers features such as presen-
tation control, home automation, background change, and sign language interpreta-
tion. The authors demonstrated a cost-effective prototype with easy installation and
extensive wireless signal transmission capabilities. The paper discusses the results,
applications, methodology, and challenges, highlighting its potential to improve
human–machine interaction across different fields.
330 V. P. Rajanala et al.

Damdoo et al. [12] presented a vision-based adaptive hand gesture recognition

system employing Convolutional Neural Networks (CNN) for machine learning clas-
sification. The study addresses the challenges of recognizing dynamic hand gestures
in real-time and focuses on the impact of lighting conditions. The authors highlight
that the performance of the system significantly depends on lighting conditions, with
better results achieved under good lighting. They acknowledge that developing a
robust system for real-time dynamic hand gesture recognition, particularly under
varying lighting conditions, is a complex task. The paper offers insights into the
potential for further improvement and the use of filtering methods to mitigate the
effects of poor lighting, contributing to the field of dynamic hand gesture recognition.
Rutika et al. [13] presented a real-time hand gesture recognition system for effi-
cient human–computer interaction. It allows remote control of PowerPoint presenta-
tions through simple gestures, using Histograms of Oriented Gradients and K-Nearest
Neighbor classification with around 80% accuracy. The technology extends beyond
PowerPoint to potentially control various real-time applications. The paper addresses
challenges in creating a reliable gesture recognition system and optimizing lighting
conditions. It hints at broader applications, such as media control, without interme-
diary devices, making it relevant to the human–computer interaction field. References
cover related topics like gesture recognition in diverse domains.

26.3 Methodology

The project’s primary objective is to make the presentation easy for the presenter to
deliver comfortably by controlling the complete presentation through hand gestures.
The whole concept of this project is demonstrated in Fig. 26.1. It gives a complete
step-by-step process from uploading of files to till termination of the presentation.

Fig. 26.1 Cyclic process

26 Hand Gesture Control for Automated Digital Presentations 331

26.3.1 Data Collection

In this project, the input data is given by the user in the form of ppt slides in images
format where the user will convert the ppt slides into images and those images will
be stored in a folder. The folder with images is the data for this project, specified in
Fig. 26.1.

26.3.2 Data Preprocessing

To rename and organize a set of PNG images, the initial step involves assigning
sequential numbers to them in the desired order. This can be achieved through
scripting or batch operations using programming or command-line tools. Once
renamed, the images will have consecutive identifiers, making it easier to organize
and retrieve them in a logical order.
After successfully renaming the PNG images with sequence numbers, the next
step is to sort them based on these assigned numerical values. Sorting ensures that
the images are used in the correct order, following the numerical sequence. This
process is crucial when creating presentations (PPT) or when a specific order is
required for image usage, as it ensures that the images are in the desired sequence for
easy access and presentation purposes. Overall, these procedures simplify the task
of organizing and working with PNG images in a structured and orderly manner.
After uploading the files folder, the data preprocessing starts renaming the images
and sorting immediately and storing them back in the folder takes place as shown in
Fig. 26.2.
Hand Detection: The method recognizes and localizes a hand’s position within a
video frame. Hand detection is the key objective in this research, and we employed
the Kanade-Lucas-Tomasi (KLT) algorithm to identify and locate all known objects
in a scene [14]. The algorithm starts by identifying feature points in the first frame

Fig. 26.2 Preprocessing of data

332 V. P. Rajanala et al.

of a video or image sequence. These features could include corners, edges, or any
other distinguishing points in the image. The Harris corner detector [15] is commonly
used for feature detection. It detects corners by analyzing intensity changes in various
directions. Once the features are identified in the first frame, the algorithm attempts
to track them in subsequent frames. It is assumed that the features move in small
steps between frames.
A small window is considered around each initial frame feature point. The algo-
rithm searches the next frame for the best window match. Feature point optical flow
is estimated using the Lucas-Kanade method [10]. The motion is assumed to be
constant in a local neighborhood around the feature point. The optical flow equation
is solved for each window pixel around the feature point. Motion parameters (w) and
spatial intensity gradients (Ix and Iy ) are related by this equation. The KLT algorithm
analyzes spatial gradient matrix as specified in Eq. (26.1), eigenvalues to determine
feature point tracking reliability. Spatial gradients of intensity in the window around
the feature point determine the matrix. A feature point is reliable for tracking if its
matrix eigenvalues are above a threshold. Figure 26.3 describes the tracking of the
hand with the help of matrix eigenvalues.
 
 w2 Ix2  w2 Ix Iy  w2 Ix Iy  w2 Iy2 (26.1)

Finger Tracking: After detecting the hand, the algorithm records the location of
individual fingers. It may entail estimating hand landmarks to pinpoint crucial spots
on the fingers, particularly the fingertips.
Finger State Classification: The algorithm defines each finger’s state as “up” (1) or
“down” (0) based on its location and movement. To establish these classifications, it
most likely evaluates the angles and placements of the fingers compared to a reference
hand form.
Finger State Combination: The algorithm creates a combination of finger states for
the entire hand. For instance, if all fingers are labeled “up,” it may indicate “5”. If
all the fingers are marked “down,” it may indicate “0.”

Fig. 26.3 Hand track mechanism

26 Hand Gesture Control for Automated Digital Presentations 333

26.4 Results

The hand tracking mechanism, finger state classification, and combination allow
each finger to be identified and assigned to a specific task. Figure 26.4 depicts this
classification for the purpose of presentation. The first gesture is used to move the
slide to the previous slide, the second gesture is used for the next slide, the third one
is used for the pointer to point the object on the slide, the fourth gesture is used to
delete the object drawn with the help of the fifth gesture, and the final gesture is used
to exit the presentation.
There were several experiments that we carried out in order to assess the effective-
ness of the system. The first experiment was designed to determine how accurate the
detection and classification of hand gestures turned out to be. We discovered that the
system was able to accurately detect and categorize hand gestures in most situations.
Figure 26.5 shows the hand tracking and gesture accuracy with the help of the KLT
algorithm of the system, an accuracy rate of approximately 95%. We conducted a
second experiment in which we examined the system’s capability of controlling a
presentation with hand gestures. It was discovered by us that the system was able
to control the slides in a smooth manner and carry out a variety of actions, such as
moving forward or going back to the slide that came before it.
In the current model, we simply set the gesture array using the built-in Hand
Tracking Module, saving time on training, and collecting hand gestures. Converting
PowerPoint to images and uploading them will take very little time. The accuracy of
the built-in model ranges from 95 to 97%. The previous model required more time
for hand tracking because there was no built-in model for detecting hand gestures,
and the accuracy was <95%.
For this project HD camera is mandatory, the range of normal inbuilt cameras in
existing laptops is 5 m. To get a long range of gesture recognition we need to use
external long-range cameras. Once the termination gesture is used the files will be
deleted. If the user wants to use the files again then they should upload the files again.

Fig. 26.4 Gestures to control the presentation

334 V. P. Rajanala et al.

Fig. 26.5 Accuracy And Frame Analysis of KLT Algorithm

26.5 Conclusion

This project is an innovative and interactive presentation control system that utilizes
computer vision and gesture recognition. It offers a hands-free and engaging way
to interact with your presentation slides. With the ability to control slide naviga-
tion through specific hand gestures, such as moving to the next or previous slide,
your project provides a convenient and intuitive alternative to traditional clickers or
keyboard shortcuts. Additionally, the option to write on the slides and erase content
by making hand movements enhances the interactivity of your presentations. The
pointer highlighter feature allows you to draw attention to specific details on your
slides, making it a powerful tool for emphasizing key points.
Furthermore, the capability to terminate the presentation with a gesture provides
an efficient way to conclude your talk. Overall, your project empowers presenters to
connect with their audience more dynamically and engagingly, all while using the
OpenCV interface and hand-tracking technology. It is a valuable addition to the realm
of presentation tools, enabling more interactive and captivating communication. The
future enhancement for this will be adding voice commands along with the hand
gesture and presenter recognition.
The challenges in this are that we are currently using 6 gestures to control the
presentation, but we can improve it by adding more gestures. We intend to add speech
commands to perform operations such as moving the slides back and forth.

References

1. Lawrence, D.O., Ashleigh, M.J.: Impact of Human-Computer Interaction (HCI) on Users in

Higher Educational System: Southampton University as a Case Study, vol. 6, no. 3, pp. 1–12,
September (2019)
2. Raschka, S., Patterson, J., Nolet, C.: Machine Learning in Python: Main Developments and
Technology Trends in Data Science, Machine Learning, and Artificial Intelligence (2020)
26 Hand Gesture Control for Automated Digital Presentations 335

3. Zhai, X., Chu, X., Chai, C.S., Jong, M.S.Y., Istenic, A., Spector, M., Liu, J.-B., Yuan, J., Li,
Y.: A review of artificial intelligence (AI) in education from 2010 to 2020 (2021)
4. Jadhav, D., Lobo, L.M.R.J.: Hand gesture recognition system to control slide show navigation.
IJAIEM 3(4) (2014)
5. Ren, Z., et al.: Robust part-based hand gesture recognition using Kinect sensor. IEEE Trans.
Multimedia 15(5), 1110–1120 (2013), pp. 8–11
6. Devivara Prasad, G., Srinivasulu, M.: Hand gesture presentation by using machine learning.
IJIRT 9(4) (2022)
7. Reethika, G., Anuhya, P., Bhargavi, M.: Slide presentation by hand gesture recognition using
machine learning. IRJET 10(01), January 2023 (2023)
8. Khanum, H., Pramod, H.B.: Smart presentation control by hand gestures using computer vision
and Google’s Mediapipe. IRJET 9(07), July 2022 (2022)
9. Powar, S., Kadam, S., Malage, S., Shingane, P.: Automated digital presentation control using
hand gesture technique. ITM Web Conf. 44, 03031 (2022)
10. Gokul, L.P., Adarsh, P., Vinayan, G., Ponmalar, M., Aswini, S.H.: Lucas Kanade based optical
flow for vehicle motion tracking and velocity estimation. In: International Conference on
Control, Communication and Computing (ICCC), Thiruvananthapuram, India, pp. 1–6 (2023).
https://doi.org/10.1109/ICCC57789.2023.10165227
11. Paulson, M., Nathasha, P.R., Davis, S., Varma, S.: Smart Presentation Using Gesture
Recognition, vol. 2, no. 3 (2017)
12. Damdoo, R., Kalyani, K., Sanghavi, J.: Adaptive hand gesture recognition system using
machine learning approach. Biosci. Biotech. Res. Comm. Spec. Iss. 13(14), 106–110 (2020)
13. Rutika, B., Shweta, C., Shraddha, D., Auti, M.A.: Powerpoint presentation control using hand
gestures recognition. Int. J. Res. Publ. Rev. 4(5), 5865–5869, May 2023 (2023)
14. Mikhaylov, D., Samoylov, A., Minin, P., Egorov, A.: Face detection and tracking from image
and statistics gathering. In: 2014 Tenth International Conference on Signal-Image Technology
and Internet-Based Systems, Marrakech, Morocco, pp. 37–42 (2014). https://doi.org/10.1109/
SITIS.2014.85
15. Zhao, J., Su, L., Wang, X., Li, J., Yang, F., Jiang, N., Hu, Q.: DTFS-eHarris: a high accuracy
asynchronous corner detector for event cameras in complex scenes. Appl. Sci. 13, 5761 (2023).
https://doi.org/10.3390/app13095761
16. Wahid, M.F., Tafreshi, R., Al-Sowaidi, M., Langari, R.: An efficient approach to recognize
hand gestures using machine-learning algorithms. In: 2018 IEEE 4th Middle East Conference
on Biomedical Engineering (MECBME), Tunis, Tunisia, pp. 171–176 (2018). https://doi.org/
10.1109/MECBME.2018.8402428
17. Talele, A., Patil, A., Barse, B.: Detection of real-time objects using TensorFlow and OpenCV.
Asian J. Converg. Technol. 5 (2019)
18. Al Saedi, A.K.H., Al Asadi, A.H.H.: A new hand gestures recognition system. Indonesian J.
Electr. Eng. Comput. Sci. 18 (2020)
19. Raschka, S., Patterson, J., Nolet, C.: Machine learning in Python: main developments and
technology. Trends Data Sci. Mach. Learn. Artif. Intell. (2020)
20. Dhall, I., Vashisth, S., Aggarwal, G.: Automated hand gesture recognition using a deep convolu-
tional neural network. In: 10th International Conference on Cloud Computing, Data Science &
Engineering (2020)
21. Paulson, M., Natasha, P.R., Davis, S., Varma, S.: Smart presentation using gesture recognition
and OpenCV. Asian J. Converg. Technol. 5 (2019)