Intelligent Systems with Applications 19 (2023) 200262

Contents lists available at ScienceDirect

Intelligent Systems with Applications

journal homepage: www.journals.elsevier.com/intelligent-systems-with-applications

Camera-based interactive wall display using hand gesture recognition

Rida Zahra a, Afifa Shehzadi b, Muhammad Imran Sharif c, Asif Karim d, *, Sami Azam d,
Friso De Boer d, Mirjam Jonkman d, Mehwish Mehmood e
a
Department of Computer Science, University of Wah, Wah, Pakistan
b
Department of Computer Science, University of Lahore, Sargodha, Pakistan
c
Department of Computer Science, Kansas State University, Manhattan, USA
d
Faculty of Science and Technology, Charles Darwin University, NT, Australia
e
School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast, Belfast, United Kingdom

A R T I C L E I N F O A B S T R A C T

Keywords: the recognition of Hand gestures has become a critical point as it is widely used in everyday applications. the
Human-computer interaction challenge in this is to improve the recognition effect and develop a fast recognition method. Glove and led-based
Euclidean distance methods involve external devices in detecting and interpreting hand gestures, making human-computer inter­
Convexity hull
action less natural. So, different approaches have been used previously that use purely hand gestures in many
Bare hand gestures
systems based on human-computer interaction. This system provides a more natural human-computer interac­
tion; it must be made efficient processing speed of classifying the test data (images) from among the training data
(database stored for gestures recognition). This speed makes gesture recognition more effective and reliable to
use as compared to previously proposed methods. In this research paper, a proposed system based on a camera-
based interactive wall display using bare hand gestures with efficient processing speed for controlling the speed
of the mouse and other functions. This system has three modules: one uses Genetic Algorithm and Otsu
thresholding to identify the query images as the right or wrong gesture and perform the correct action in case of
the proper motion, another module controls functions outside of PowerPoint files or Word documents, e.g., to
open folders and go through drives, and the third module uses the convexity hull method for finding the number
of fingers open in the user’s gesture and operates accordingly.

1. Introduction and computer systems (Jiang et al., 2023). Hand gesture recognition
technology has numerous practical applications in various fields,
A range of remarkable developments in the field of hand gesture including gaming, virtual reality, healthcare, and education (Maske­
recognition technology has been seen in recent years (Damaneh et al., liūnas et al., 2023). For instance, in healthcare, this technology can
2023). This technology has revolutionized human-computer interaction, control medical equipment through hand gestures, allowing surgeons to
allowing users to operate computer systems through intuitive and nat­ operate without physical contact, thus reducing the risk of infections.
ural hand gestures. Hand gesture recognition involves several processes, With the use of cutting-edge technology in education, interactive
which include image capture, hand detection, hand extraction, and teaching aids can be created using hand gesture recognition technology,
gesture classification (Wu et al., 2023; Fayyaz et al., 2023). These pro­ increasing learning effectiveness and engagement. (Dimitriadou &
cesses have become increasingly easy to use, enabling users to interact Lanitis, 2023). However, the accurate detection and extraction of the
easily with computers and other devices. There are various methods for hand from an image or video feed remain one of the primary challenges
hand gesture recognition, including glove and led-based methods (Lin of hand gesture recognition (Sharif et al., 2022; Cohen et al., 2022). This
et al., 2022; Singh & Chaturvedi, 2022). However, these methods requires sophisticated algorithms to identify the hand based on shape,
require external devices that could interfere with the naturalness of the color, and movement (Alyami et al., 2023). Machine learning algorithms
human-computer interaction (Bi et al., 2023). As a result, researchers often classify hand gestures based on a predefined set of gestures. The
have developed techniques that merely use hand gestures to control advances in deep learning algorithms and the availability of large
computer functions, enabling more natural interaction between users datasets have significantly contributed to the advancement of gesture

* Corresponding author.
E-mail address: [email protected] (A. Karim).

https://doi.org/10.1016/j.iswa.2023.200262
Received 15 May 2023; Received in revised form 18 July 2023; Accepted 30 July 2023
Available online 1 August 2023
2667-3053/© 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-
nc-nd/4.0/).
R. Zahra et al. Intelligent Systems with Applications 19 (2023) 200262

recognition technology (Ghosh et al., 2021). These advancements have hand gestures. The proposed system uses left-hand gestures for music
made the technology more accessible and user-friendly, enabling users control. It mainly helps to differentiate between different gestures for
to interact with computers and other devices more naturally and intui­ instance pointing left, pointing right, shaking hand, moving up and
tively (Morajkar et al., 2023; Islam Chowdhury et al., 2020). This down, etc. Lin et al. (2012) propose a method for hand gestures. First,
research aims to enhance human-computer interaction further. This the hand is tracked using a skin model that uses color information. A
system projects the system’s screen image onto a wall and is used as a single Gaussian model differentiates between skin colors and the hand
computing platform to support gesture interaction between humans and region gets identified. Hand gestures are detected using saliency point
computers. This system can make presentations more accessible by hand detection. Two points such as edge points and inner points get detected
gestures interacting with the display wall. The mouse cursor is also using this method. Zhou et al. (2016), proposed a system that can ab­
controlled, and various operations can be performed by hand gestures in stract the gesture of the extensional finger and fluxional finger with high
front of the camera. It increases users’ interest and convenience. Section accuracy. The proposed algorithm takes no effect on the results by hand
2 of this research paper includes an analysis of related prior work on rotation and finger angle variation Lahiani and Neji (2018) presented an
human-computer interaction. Section 3 outlines the suggested approach. investigation for improving hand gesture detection, particularly for
The experimental results of various hand gestures of a system and their mobile devices. They presented a technique for static hand gesture
rates of recognition are displayed in Section 4. Section 5 gives a recognition for the mobile system. In this system, they combined the
conclusion and describes how the current approach is appropriate and histogram of oriented gradients (HOG) and (LBP). The combined feature
easy to use compared to previous methods. achieved a detection rate of 92% accuracy. Shanthakumar et al. (Barb­
The proposed novelty of this paper is a camera-based interactive wall huiya et al., 2021) proposed a novel angular velocity model. The model
display system that uses bare hand gestures for human-computer works in real-time capturing the 3D motion data through the
interaction without requiring external devices. The system is designed sensor-based system. The proposed model captures both static and dy­
to have efficient processing speed for classifying test data (images) from namic data in real-time for hand gesture recognition. Calado et al.
among the training data stored in a database for gesture recognition, (2022) presented a model which is based on geometric approaches. This
making gesture recognition more effective and reliable than previously method supports visualization and geometrical interpretation for
proposed methods. The system comprises three modules, including one gesture recognition.
that uses Genetic Algorithms and Otsu thresholding for identifying the Baumgartl et al. (2021) proposed a technique to detect simple hand
query images as the right or wrong gesture and performing the correct gestures which are used in a variety of. the technique was based on a
action, another module that controls functions outside of PowerPoint convolutional neural network on MobileNetV2. The combination pro­
files or Word documents, and the third module that uses the convexity duced an accuracy of 99.69% in detecting the hand gesture precisely.
hull method to find the number of fingers open in the user’s gesture and Alnuaim et al. (2022) presented a model to detect hand gestures used in
operate accordingly. sign language. The system was composed of two convolutional neural
network models and both were trained separately on the dataset of
2. Related work Arabic sign language. The final results of both models were combined to
achieve more accurate results. the hand gesture images were first pro­
There are numerous hand gesture recognition techniques have been cessed and then fed into the models which were ResNet50 and Mobile­
proposed and implemented before. Viola Jones method (Yun & Peng, NetV2. The models achieved an accuracy of 97%. Xu et al. (2022)
2009) is a well-known technique for hand gesture detection. This presented a technique to detect hand gestures when it is static or dy­
method can accurately detect hand regions by locating them with a blue namic, for the static hand gesture detection recognition the palm is
rectangle. For feature extraction, it incorporates a set of Haar-like fea­ detected by contouring the hand gesture image and applying the Dis­
tures into the image representation. Xu et al. (2009) presented a method tance Transform Algorithm. For the localization of finger K- The
that uses a vision-based hand-tracking approach. This approach is curvature-Convex Defect Detection algorithm was used. A recognition
mainly used for Hand gesture segmentation. A self-organizing map algorithm was presented for static gesture recognition, and for dynamic
(SOM) is another technique, for hand gesture recognition using 5DT data hand gesture recognition Improved Dynamic Time Warping algorithm
gloves, proposed by Jin et al. (2011). This technique is a machine was used. Chua et al. (2022) presented a system that uses object
learning algorithm in which raw data is used as input vectors and then detection techniques combined with handcrafted rules for recognizing
plots a mapping between this data and gesture commands. A dynamic hand gestures on a computer. The system was based on the
camera-based handwriting gesture recognition system is used for YoloV3 and RGB cameras for the localization and recognition of hand
hand-held devices like mobile phones. Edge detection is implemented gestures. A self-designed database was used and it achieved an accuracy
for each frame and edge histograms are plotted. The method proposed of 96.68%. Azim et al. (2023) presented a tool for the recognition of foot
by Gu et al. (2012) has two modules, one for enhanced TSL self-adaptive gestures when hands are unavailable. The tool can produce a gesture set
hand area detection and the other for contour signature-based finger that helps produce the data set of foot gestures. It works by dividing the
detection. It uses employs the contour signature technique. In this gesture into atomic actions. The tool gives a very low false positive,
technique, the distance sequence from contour points and moment de­ tested on nine datasets of gesture designers. the use of intelligent sys­
termines fingertip points. Using a threshold box in HSV color space, the tems is very common these days for the detection and segmentation of
hand area is roughly segmented. Bainbridge and Paradiso (2011) pro­ objects within the image Calisto et al. (2022). Intelligent agents are fast
posed a wearable hand gesture recognition system that detects hand and have been very successful with high true prediction rates. Vatavu
gestures. RFID (radio frequency electromagnetic fields) is used to detect (2023) introduces a taxonomy for index- Finger Augmentation Devices
hand gestures. Tags or sensors are attached to the fingers and are (iFAD). It was tested on a dataset composed of 6369 gestures gathered
identified using the electromagnetic field. Panwar (2012) proposed from 20 participants. It has been seen in the results that the iFAD is very
another technique in which the hand is detected through shape pa­ fast obtaining an average of 1.84 s. The study showed that the iFAD
rameters. K-means clustering is used to extract hand from the image. gestures are twice faster as compared to the other gestures produces by
Trindade et al. (2012) proposed a technique in which color and to detect the body parts. The hand gesture could be found by the techniques used
hand gestures depth images are used. The hand region is filtered out to detect the objects with assertive and non-assertive tones also, as used
using a color filter. The sensor is used on the palm that detects hand by Calisto et al. (2023). The researcher used it for the detection of tu­
pose. The hand’s center is formulated using the k-means algorithm. mors and the method received promising results. Kumar and Pandian
Hand gesture is detected using a depth histogram. Modler & Myatt (2023) used the Convolutional Deep VGG-16 (CDVGG-16) for the
(2008) propose a method in which sound and music are controlled using recognition of sign language which uses hand gestures to produce sign.

2
R. Zahra et al. Intelligent Systems with Applications 19 (2023) 200262

Fig. 1. System architecture.

The blocks used in the system were composed of 2D Convolution and image matches with any of the dataset images, then the corresponding
Max pooling Layers. For the feature extraction VGG-16 was used. The OS operation is performed. Binary images are used as dataset images as
Sign Language model (SLR) was introduced with CDVGG-16 classifier. well as test frames. All the processing is performed on binary images.
The proposed framework produced an accuracy of 95.6% accuracy. The size of real-time captured frames is maintained according to the
Most of these techniques used hardware-based systems for hand dataset images. The camera used for the system is compliant with USB
gesture recognition. this makes human-computer interaction difficult 2.0 Interface, Compatible with MS Windows XP Service Pack 3, Auto­
and uneasy. Also in poor light conditions, hand areas were not detected focus system, Up to 2-megapixel resolution, Color depth: 24-bit true
correctly. Most of the applications have not used user-friendly interfaces color or above, Video capture: Up to 1600 × 1200 pixels (HD quality).
and needed training to use the application. In this article, a method has With the auto focus capability, the camera captured the sharp image
been proposed that overcomes these difficulties to some extent and focusing the object. The camera could handle the dim lightning as well
enables the user to use hardware-free hand gestures to control opera­ as bright light. However dim lightning could introduce noice and blur.
tions of Powerpoint files, and Word documents and to control mouse Bright light is best if it is not the sun light. In case of mix light it uses the
cursor functions more naturally and interactively. Also, the user has to autofocus for the best result. Two major modules of the system are
just start the application and control systems operations through hand Gesture Validation and Cursor Control. The steps of the gesture valida­
gestures. The user does not need training for the proper operation of the tion module are shown in the following flowchart in Fig. 2.
application.
3.1. Video capturing and frames formation
2.1. Research problem
A real-time capturing device like a webcam or external USB-
This research makes use of AI and Image processing techniques to connected camera is used for recording the video. Video is converted
enable the users to control the cursor on the computer screen for into frames. Each frame is captured by an interval of 2 s. Each frame is
browsing the files and folders and performing left and right-click oper­ captured and processed side by side.
ations with the index finger. To operate PowerPoint presentations with
specified gesture movements and scroll pdf documents up and down
3.2. Frames preprocessing
with separate types of gestures. And Genetic algorithm helps to build a
speedy way of classifying the test data (images) from among the training
Frames are preprocessed before applying recognition techniques.
data (database stored for gestures recognition).
The frame is resized to the size of the dataset images. Filters are applied
to remove noise from the image. And finally, a clear image is obtained.
3. Proposed methodology

The methodology proposed by a given system is shown by the system 3.3. Hand detection
architecture in the following Fig. 1.
Hand gestures are captured and recorded as single frames. These For hand detection, firstly the contrast function is applied to enhance
frames are passed for preprocessing like smoothing and resizing. The skin color, the skin detection algorithm is then executed. From the
hand detection module is then given the input image. Then hand captured frame, skin is detected for a specific YGBR value. The system is
extraction is performed for cropping the hand region from the image. then trained to detect lighter to darker skin colors. After skin detection,
Now the image is ready for gesture validation. If the gesture of the test the entire portion other than the skin areas is removed from the image.

3
R. Zahra et al. Intelligent Systems with Applications 19 (2023) 200262

Fig. 5. Extracted hand gesture.

Fig. 6. Hand gesture dataset.

3.4. Hand extraction

The hand extraction is performed based on the connected component

algorithm. The connected component algorithm takes the largest con­
nect part from the image which is always the hand region. The hand
region is cropped out of the image in this manner, and the rest of the
image is eliminated. Now the obtained image is ready for validation
(Figs. 3 and 4).

3.5. Hand gesture validation

Once the hand area has been extracted from the image, it is then
compared with all the images present in the dataset through the
Euclidean distance method. First of all user’s hand image is converted
into a binary image using otsu’s method that helps to make the back­
ground of an image uniform as shown in Fig. 5.
All dataset images are stored in binary form. Euclidean distance
among the user’s hand gesture image and dataset images is found by
subtracting every pixel of the user’s hand image from the respective
pixels of all images in the dataset. The Euclidean distance formula is
Fig. 2. Gesture validation flow chart. shown below.
Euclidean = (float)(Math.Sqrt((clr − clr1) ∗ (clr − clr1)));
Where clr is captured frame pixel and clr1 is the dataset image pixel.
Among all dataset images, the image that has the smallest Euclidean
distance with the user’s hand gesture image, will be the final match for
the given hand gesture, and hence corresponding operation will be
performed against that gesture e.g. scroll up, scroll down, close .ppt or .
word file. For single-hand gestures, the dataset has similar gesture im­
ages from various angles to provide the best match for a real-time
captured frame. Fig. 6 shows some of the samples.

3.6. Algorithms including AI

Fig. 3. Video frame. We used an improved Genetic Algorithm and Otsu thresholding to
identify the query images as the right or wrong gesture and perform the
correct action in case of the right gesture. The genetic algorithm helps to
build a speedy way of classifying the test data (images) from among the
training data(database stored for gesture recognition). This speed makes
gesture recognition more effective and reliable to use in this research as
compared to previously proposed methods.

3.7. Genetic algorithms

To find the best solution genetic algorithm used the technique of

Fig. 4. Extracted hand gesture. finding the fittest individual from the pool of the population. In the

4
R. Zahra et al. Intelligent Systems with Applications 19 (2023) 200262

3.8. Otsu thresholding

To calculate the measure of the spread for the pixels, the Otsu
threshold method is used. This is done so pixels on either side of the
threshold are measured, the foreground and background. The main aim
is to find the threshold value which has a minimum spread sum of the
foreground and background. A 6 × 6 image demonstration is shown
below in Fig. 7.

3.9. Cursor control

The cursor control flow is shown in Fig. 8:

Fig. 7. A 6-level greyscale image & its histogram.

3.10. Fingers count approach
initial state, a population of the possible solution is given to the problem.
In this project, the mouse cursor was controlled through hand ges­
The provided solutions are more refined by recombination and mutation
tures to perform the click function to open folders and .ppt and .doc files.
methods as in the natural genetic process. This results in producing new
Also, the mouse cursor was moved with hand motion by finding the
children and it continues over various generations. A fitness value that is
Center of the palm. To control mouse clicks through hand gestures, the
based on the objective function is given to the function. The fitter the
first skin part is detected to extract the hand area from real-time video.
candidate the more is the chance to pass on to the next generation.
For this purpose YCbCr filter was used that helped to get skin areas.
Erosion and dilation of the image are done after skin detection. Gaussian
filter is applied to filter the image. The most significant portion of the
skin area is always the hand detected by finding the most prominent

Fig. 8. Cursor control flowchart.

5
R. Zahra et al. Intelligent Systems with Applications 19 (2023) 200262

for the current gesture database. Fig. 9 shows soe images of finger
counts.

4. Experimental results

A comprehensive real-time experimentation is critical for such sys­

tems as demonstrated by multiple studies I this field (Miah et al., 2023;
Calisto et al., 2017; Schak & Gepperth, 2022). Therefore, a complete
real-time performance of hand gestures has been evaluated in this sec­
tion, and the results have been shown in Tables 1 and 2. 60 real-time
images of each hand gesture have been processed and tested. Some­
times motion’s required function was not performed due to invalid
lightning conditions or inappropriate performance of gestures. Howev­
er, the table results show that the correct performance percentage
occurred between 90% and 98%, which is a suitable test result for this
application. Table 1 shows hand gestures when the user’s hand move­
ment controls the cursor. The first gesture was with all fingers closed and
the action mapped against it is system shut down. 56 images were
validated by the system out of a total of 60. The gesture received 93.33%
recognition rate. The second gesture was with index finger, opened for
controlling the mouse cursor. 55 images were validated by the system
from an input set of 60 images and it achieved recognition rate of
91.66%. The gesture with index and middle fingers, opened for single
click produced 95% recognition rate. The gesture with index and ring
fingers opened for double click achieved 96.66% recognition rate. The
gesture with index, middle, ring and little fingers opened for right click
achieved 91.66%. The gesture with all five fingers opened for the exit
action received a recognition rate of 90%.
Table 2 shows gestures used inside currently open PowerPoint or
Word documents to perform scroll up, scroll down, and close functions.
The gestures overall performed well on Power Point and Word file as
Fig. 9. Finger count. compared to previous for the system receiving high recognition rate for
some gestures.

contour. Convexity defects are used to count the number of fingers in the
4.1. Average recognition rate: 93.7%
user’s gesture image. For instance, the exit function is performed if the
finger count is five. One "move cursor" function is performed when the
4.1.1. Overall average recognition rate: (93+93.72)/2 = 93.35%
finger count is.
Our proposed systems achieved a higher accuracy rate as compared
Similarly, for finger count two, three, and four, a single operation of
to the other existing systems. Table 3 presents the comparison of existing
a single click, double click, and right click are performed, respectively.
techniques with our system.
The Center of the palm is detected to control cursor movement. For this
purpose, coordinates of the Center of the contour were found. The mouse
5. Conclusion
cursor is controlled by hand by finding the contour center. In case of an
invalid gesture, the cursor will not move or perform the click function.
In this study, a method has been proposed to recognize hand gestures
The system is capable to train on more hand gestures if new hand ges­
and operate their respective functions. We used an improved Genetic
tures are inserted in the system. it will work the same way as it is doing
Algorithm and Otsu thresholding to identify the query images as the

Table 1
Test results of hand gestures for mouse cursor control.
Gesture Name Input images Validated images Recognition rate= (Input/validated)*100 Function Against Gesture

Gesture with all fingers closed 60 56 93.33 System Shut down

Gesture with index finger open 60 55 91.66 Mouse cursor control
Gesture with index and middle finger open 60 57 95 Single click
Gesture with index, middle and ring finger open 60 58 96.66 Double click
Gesture with index, middle, ring and little finger open 60 55 91.66 Right click
Gesture with all five fingers open 60 54 90 Exit

Table 2
Test results of hand gestures for functions of powerpoint and word files.
Gesture Name Input images Validated Images Recognition rate (%) Function Against Gesture

Gesture with only index and small finger open 60 59 98.33 Show slide show of PowerPoint
Gesture with all fingers closed 60 54 90 Scroll down to the next slide or page
Gesture with only thumb open 60 55 91.66 Scroll up to the previous slide or page
Gesture with all five fingers open 60 57 95 Close the currently open file or explorer window

6
R. Zahra et al. Intelligent Systems with Applications 19 (2023) 200262

Table 3 References
Comparison table.
Alnuaim, A., et al. (2022). Human-computer interaction with hand gesture recognition
S# Research Paper Recognition Rate
using ResNet and MobileNet. Computational Intelligence and Neuroscience, 2022.
1. Yun and Peng (2009) 85% Alyami, J., et al. (2023). Tumor localization and classification from MRI of brain using
2. Xu et al. (2009) 91% deep convolution Neural Network and Salp Swarm Algorithm. Cognitive Computation,
3. Jin et al. (2011) 91% 1–11.
Azim, M. A. R., Rahman, A., & Heo, S. (2023). SequenceSense: A tool for designing usable
4. Gu et al. (2012) 91%
foot-based gestures using a sequence-based gesture recognizer. International Journal
5. Lahiani and Neji (2018) 92%
of Human-Computer Studies, 176, Article 103035.
6. Given experiment 93.35% Bainbridge, R., & Paradiso, J. A. (2011). Wireless hand gesture capture through wearable
passive tag sensing. In Proceedings of the 2011 international conference on body sensor
networks. IEEE.
Barbhuiya, A. A., Karsh, R. K., & Jain, R. (2021). CNN based feature extraction and
classification for sign language. Multimedia Tools and Applications, 80(2), 3051–3069.
right or wrong gesture and perform the correct action in case of the Baumgartl, H., et al. (2021). Vision-based hand gesture recognition for human-computer
interaction using MobileNetV2. In Proceedings of the 2021 IEEE 45th annual
proper motion. The genetic algorithm helps build a speedy way of
computers, software, and applications conference (COMPSAC). IEEE.
classifying the test data (images) from the training data (database stored Bi, S., et al. (2023). Flexible pressure visualization equipment for human-computer
for gesture recognition). This speed makes gesture recognition more interaction. Materials Today Sustainability, Article 100318.
effective and reliable in this research than previously proposed methods. Calado, A., et al. (2022). A geometric model-based approach to hand gesture recognition.
IEEE Transactions on Systems, Man, and Cybernetics: Systems, 52(10), 6151–6161.
Video of hand gestures were captured through a camera. Video frames Calisto, F. M., Ferreira, A., Nascimento, J. C., & Gonçalves, D. (2017). Towards touch-
are then preprocessed to improve the quality of structures. The contrast based medical image diagnosis annotation. In Proceedings of the 2017 ACM
function was applied to the captured frames to enhance the color of skin international conference on interactive surfaces and spaces. https://doi.org/10.1145/
3132272.3134111
pixels. The hand’s area was extracted using skin detection and a con­ Calisto, F. M., Nunes, N., & Nascimento, J. C. (2022). Modeling adoption of intelligent
nected component algorithm. The best match between the images from agents in medical imaging. International Journal of Human-Computer Studies, 168,
the dataset and the captured frame was determined using the Euclidean Article 102922.
Calisto, F. M., et al. (2023). Assertiveness-based agent communication for a personalized
distance formula. The dataset image with the shortest distance from the medicine on medical imaging diagnosis. In Proceedings of the 2023 CHI conference on
captured frame is considered to be more similar and validated against human factors in computing systems.
the captured frame. The mouse cursor was also controlled using the Chua, S. D., et al. (2022). Hand gesture control for human–computer interaction with
Deep Learning. Journal of Electrical Engineering & Technology, 17(3), 1961–1970.
finger count method. The proposed process does not involve hardware Cohen, J. E., et al. (2022). Low-income and middle-income countries leading the way
like data gloves or LED lights for hand gesture detection. The proposition with tobacco control policies. BMJ Innovations, 8(1), 4–8.
provides a more natural way to communicate with computer systems. Damaneh, M. M., Mohanna, F., & Jafari, P. (2023). Static hand gesture recognition in
sign language based on convolutional neural network with feature extraction
The technique performs well and takes less amount of time to validate
method using ORB descriptor and Gabor filter. Expert Systems with Applications, 211,
gestures. It is also user friendly where users do not require any special Article 118559.
training to operate it. The users just have to perform motions in front of Dimitriadou, E., & Lanitis, A. (2023). A critical evaluation, challenges, and future
the camera, and their respective functions will be served. perspectives of using artificial intelligence and emerging technologies in smart
classrooms. Smart Learning Environments, 10(1), 1–26.
Fayyaz, A. M., et al. (2023). An integrated framework for COVID-19 classification based
CRediT authorship contribution statement on ensembles of deep features and entropy coded GLEO feature selection.
International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 31(01),
163–185.
Rida Zahra: Conceptualization, Methodology, Software. Afifa Ghosh, P., et al. (2021). A comparative study of different deep learning model for
Shehzadi: Conceptualization, Methodology, Software. Muhammad recognition of handwriting digits. In Proceedings of the 2020 international conference
Imran Sharif: Data curation, Writing – original draft, Writing – review on IoT based control networks and intelligent systems (ICICNIS 2020) (pp. 857–866).
https://doi.org/10.2139/ssrn.3769231. January 192020.
& editing, Visualization, Investigation. Asif Karim: Data curation, Gu, L., Yuan, X., & Ikenaga, T. (2012). Hand gesture interface based on improved
Writing – original draft, Writing – review & editing, Supervision. Sami adaptive hand area detection and contour signature. In Proceedings of the 2012
Azam: Conceptualization, Methodology, Software, Supervision. Friso international symposium on intelligent signal processing and communications systems.
IEEE.
De Boer: Validation. Mirjam Jonkman: Validation. Mehwish Meh­ Islam Chowdhury, A., et al. (2020). An automated system in ATM booth using face
mood: Visualization, Investigation. encoding and emotion recognition process. In Proceedings of the 2020 2nd
international conference on image processing and machine vision.
Jiang, L., et al. (2023). SmartRolling: A human–machine interface for wheelchair control
Declaration of Competing Interest
using EEG and smart sensing techniques. Information Processing & Management, 60
(3), Article 103262.
The authors declare that they have no known competing financial Jin, S., et al. (2011). SOM-based hand gesture recognition for virtual interactions. In
interests or personal relationships that could have appeared to influence Proceedings of the 2011 IEEE international symposium on VR innovation. IEEE.
Kumar, G. M., & Pandian, A. (2023). A constructive deep convolutional network model
the work reported in this paper. for analyzing video-to-image sequences. Data & Knowledge Engineering, 144, Article
102119.
Data availability Lahiani, H., & Neji, M. (2018). Hand gesture recognition method based on HOG-LBP
features for mobile devices. Procedia Computer Science, 126, 254–263.
Lin, L., Cong, Y., & Tang, Y. (2012). Hand gesture recognition using RGB-D cues. In
The dataset of the images used can be downloaded from: Proceedings of the 2012 IEEE international conference on information and automation.
https://charlesdarwinuni-my.sharepoint.com/:u:/g/personal/asif_ IEEE.
Lin, P., et al. (2022). LED screen-based intelligent hand gesture recognition system. IEEE
karim_cdu_edu_au/ER9Z3CwJgr5KtMmRJ-AxRCYBbWQW Sensors Journal, 22(24), 24439–24448.
eid2otisGXCrZ0qQMQ?e=zClIJ0 Maskeliūnas, R., et al. (2023). BiomacVR: A virtual reality-based system for precise
human posture and motion analysis in rehabilitation exercises using depth sensors.
Electronics, 12(2), 339.
Miah, A. S., Hasan, Md. A., & Shin, J. (2023). Dynamic hand gesture recognition using
Funding multi-branch attention based graph and general deep learning model. IEEE Access,
11, 4703–4716. https://doi.org/10.1109/access.2023.3235368
Modler, P., & Myatt, T. (2008). Recognition of separate hand gestures by time-delay
The Reseach Received no External Funding.
neural networks based on multi-state spectral image patterns from cyclic hand
movements. In Proceedings of the 2008 IEEE international conference on systems, man
and cybernetics. IEEE.
Morajkar, A., et al., Hand gesture and voice-controlled mouse for physically challenged
using computer vision. Advanced engineering days (AED), 2023. 6: P. 127–131.

7
R. Zahra et al. Intelligent Systems with Applications 19 (2023) 200262

Panwar, M. (2012). Hand gesture recognition based on shape parameters. In Proceedings Vatavu, R. D. (2023). iFAD gestures: Understanding users’ gesture input performance
of the 2012 international conference on computing, communication and applications. with index-finger augmentation devices. In Proceedings of the 2023 CHI conference on
IEEE. human factors in computing systems.
Schak, M., & Gepperth, A. (2022). Gesture recognition on a new multi-modal hand Wu, H., et al. (2023). Thermal image-based hand gesture recognition for worker-robot
gesture dataset. In Proceedings of the 11th international conference on pattern collaboration in the construction industry: A feasible study. Advanced Engineering
recognition applications and methods. https://doi.org/10.5220/0010982200003122 Informatics, 56, Article 101939.
Sharif, M. I., et al. (2022). Deep learning and kurtosis-controlled, entropy-based Xu, H., et al. (2009). Real-time hand gesture recognition system based on associative
framework for human gait recognition using video sequences. Electronics, 11(3), 334. processors. In Proceedings of the 2009 2nd IEEE international conference on computer
Singh, S. K., & Chaturvedi, A. (2022). A reliable and efficient machine learning pipeline science and information technology. IEEE.
for american sign language gesture recognition using EMG sensors. Multimedia Tools Xu, J., et al. (2022). Robust hand gesture recognition based on RGB-D Data for natural
and Applications (pp. 1–39). USA: Kluwer Academic Publishers. human–computer interaction. IEEE Access, 10, 54549–54562.
Trindade, P., Lobo, J., & Barreto, J. P. (2012). Hand gesture recognition using color and Yun, L., & Peng, Z. (2009). An automatic hand gesture recognition system based on
depth images enhanced with hand angular pose data. In Proceedings of the 2012 IEEE Viola-Jones method and SVMs. In Proceedings of the 2009 second international
international conference on multisensor fusion and integration for intelligent systems workshop on computer science and engineering. IEEE.
(MFI). IEEE. Zhou, Y., Jiang, G., & Lin, Y. (2016). A novel finger and hand pose estimation technique
for real-time hand gesture recognition. Pattern Recognition, 49, 102–114.