A Machine Learning and IoT-based Anti-spoofing

Technique for Liveness Detection and Face

Recognition
2023 International Conference on Computer, Electronics & Electrical Engineering & their Applications (IC2E3) | 979-8-3503-3800-3/23/$31.00 ©2023 IEEE | DOI: 10.1109/IC2E357697.2023.10262733

Anshuman Srivastava Abhishek Tiwari

ECE Department Embedded Systems Department
JSS Academy of Technical Education Centre for Development of Advanced
Noida, India Computing (CDAC),
[email protected] Noida, India
[email protected]

Abstract—Security systems used in public and corporate impossible to analyze the fingerprints of a group of people at
sectors these days are becoming more and more biometrics the same time.
dependent. Face recognition is one of the most prominent tools
used in authentication systems. It used to involve creating a Another effective method to implement biometrics is the
digital template of the face of a person and identifying it, which
is a unique physical characteristic for each and every person.
‘Iris Scanner’. It involves scanning the eyes of a person and
But this method of biometrics is susceptible to certain threats. assessing the patterns of the colored tissue from which the iris
The system can be fooled using the image of the person or by of a person is formed [3]. The cameras that are used to detect
displaying videos or photos of the person in front of the camera. these patterns operate on near-infrared wavelengths. Being an
To overcome this problem, certain methods have been advanced technology, iris recognition has a quite limited
proposed, which include texture analysis, frequency analysis, impact over the world. There are certain limitations that these
3D face shaping, etc. The constraint of these methods is that systems possess. First of all, these systems are environment
they require high computational powers because of the length specific. It means that iris recognition systems fail to work
and complexity of the codes. These methods are suitable to be efficiently in unconstrained environments. The challenges
implemented on high-end devices, such as smartphones,
laptops, etc. The aim of this paper is to provide a method for
include low resolution, distortion, occlusion by an eyelash,
liveness detection suitable for edge devices. A method that is reflection from eyeglasses, etc. Another limitation that the iris
much more reliable and uses less computational power is recognition systems face is that due to the stretching and
proposed. Raspberry pi is used to implement the proposed wilting of eyes in old age people, these systems are more
technique which itself is an edge device. The two major susceptible to producing false results.
complications of a facial recognition system are tackled
separately by using the algorithms of eye blink detection and Talking about face recognition systems [4], it is one of the
object detection. most common types of biometrics used these days. From the
reports available through google search [5], in India, there are
Keywords—liveness detection, edge-device, object detection,
blink detection, raspberry pi. more than 124 facial recognition systems currently in use, and
the government has already spent more than INR 1464.18 Cr
I. INTRODUCTION on them. But, these systems are vulnerable to a major threat.
Biometrics has been the most trending security system A face recognition system could be fooled or misled to
during the past few decades [1]. Authentication by biometrics provide unauthorized access. This could prove to be a serious
involves identifying each individual by their intrinsic threat to the security system. An image or a video of a person
physical or behavioral traits. Both public and private sectors could be used to login into a system which would put the
have seen recent growth in the field of biometrics. The authenticity of the system at a greater risk. To overcome this
government of India has also adopted biometrics in e- difficulty, liveness detection is introduced in these systems.
governance applications such as Aadhar, driving licenses, Liveness detection basically distinguishes between an actual
etc. Significant growth in use of biometrics can be seen in face and a spoofed face. There are several proposed methods
healthcare sectors, schools, corporate offices, banks, and for liveness detection. Some of them are listed below [6].
even in consumer electronics.
• The 3D face shaping is a facial liveness detection
technique which is similar to the one used in iphones.
One of the most widely used biometrics is the fingerprint
recognition system. It involves scanning the fingerprint, • Frequency analysis which includes transforming the
which determines the patterns that a finger possesses [2]. face regions into frequency domain using Fourier
These patterns are unique to each and every person. But there Transform technique.
are certain vulnerabilities associated with this type of
• Texture analysis, which examines the texture of
biometric. Firstly, scanning and analyzing the fingers of all
different regions of the face by using Local Binary
the persons one by one could be time-consuming and thus Patterns (LBPs) and differentiates between real and
will reduce the efficiency of the system. Furthermore, spoofed faces through SVM algorithm.
scalability is also a major limitation of this type of biometric
system. Installing a number of fingerprint recognition • Variable focusing analysis which differentiates
systems in a single place to ensure the scanning and between real and fake faces after determining the
processing of the data of different individuals simultaneously change in pixel values between two consecutive
would be a very demanding task. Therefore, it is nearly frames.

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on November 22,2024 at 19:32:24 UTC from IEEE Xplore. Restrictions apply.
 Although these algorithms produce satisfactory results,
they pose a major limitation. These algorithms are According to this paper’s proposed model, the liveness of
comparatively slow in producing intended outputs in real the person is detected through ‘object detection’ and ‘blink
time on an edge device as they are computationally heavy. detection’. The only dataset required by the system is the
This means that these algorithms require high processing images and names of the authorized individuals to train the
power to be able to be executed efficiently. But since the edge model. This approach only requires to register the blinks of
devices possess limited computational power, it is difficult the person to initiate the detection and recognition.
for them to run such heavy algorithms.
Raspberry pi has been used as an edge device to implement
the proposed scheme. Being highly portable and cost- III. PROPOSED SCHEME
friendly, raspberry pi provided a better platform than any The aim of this experiment is to introduce a technique
other edge device. Raspberry pi model 4B+ is used to carry which implements Liveness detection at edge devices. The
out the experiment been used to achieve liveness detection in paper proposes a unique hybrid approach towards Facial
this ‘Face Recognition System’. The first one is ‘blink Liveness Detection on edge devices through Object detection
detection’ and the other is ‘object detection’. and Blink detection techniques using OpenCV for real-time
processing. The experiment has been tested in real time on
the Raspberry Pi hardware platform, shown in Fig. 1 which
II. RELATED WORK is a 64-bit SoC and provides 4GB of RAM. It uses an open-
A face recognition system can easily be fooled by holding source distribution software and runs on Debian based
the photograph or displaying an image/video of the person in Raspbian and NOOBS. The platform is considered as an edge
front of the camera. A method known as facial liveness device due to its small form factor and low power
detection is employed to eliminate this vulnerability of the consumption, which makes it easy to be deployed in a variety
system. Liveness detection can be achieved among edge of applications.
devices through various methods as mentioned in the above
sections. One of the most common methods to achieve The application involves trained dataset from algorithms
liveness detection among edge devices is Heuristic-based namely, 3D Face shaping, Variable focusing analysis
approaches such as ‘blink detection’. It involves capturing technique, Frequency analysis, etc. using the existing models
the person’s face or eyes and tracking the eye movement to to determine and infer the liveness of the person in real time.
ensure the user is an actual person [7]. As already known, these algorithms require high
computational power to run. These are not suitable for
Liveness detection using color texture analysis is one way implementation on an edge device due to their low processing
to develop an anti-spoofing face recognition system [8]. This power.
method differentiates between real and fake faces based on
the difference in color texture. It also involves training a
dataset for the spoofed faces to provide better results. Another
method to achieve anti-spoofing face recognition systems is
based on spatial and frequency-based descriptors [9]. The
feature descriptors combine color, texture information and
gradient magnitude to detect presentation attacks.

As mentioned in [10], an innovative method has been used

to detect spoofed faces based on the image quality. This
method is based on computing PSNR and SNR to
differentiate between live face and fake face. The values of
real faces are always higher than that of display aur printed
faces.

All the above-mentioned techniques though sound easy to

implement, these techniques involve certain demerits. Using
blink detection algorithm alone can produce satisfactory
results but there is a major loophole. It can be tricked by
displaying a video of a person blinking his eyes. The methods
mentioned in [8,9] involve maintaining large datasets while Fig. 1. Raspberry Pi 4B+ and Camera Module
the method proposed in [10] computes PSNR and SNR which
considers only the pixel-level differences between the real
The implementation uses Raspberry pi board and its related
and fake faces and not the overall semantics of the image. It
pi camera module for image detection. The proposed
can also produce false results due to the difference between
approach envisages an optimized implementation flow which
resolutions and size of the reference and reconstructed image.
can be ported on a limited compute hardware platform also ,
which is one of the major requirements for such an
application on edge devices.

Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on November 22,2024 at 19:32:24 UTC from IEEE Xplore. Restrictions apply.
 In the proposed method, the object detection algorithm According to the proposed scheme, if the presence of any
and blink detection algorithms are integrated with the face cell phone device is detected, the program would be
recognition algorithm. This hybrid approach to Object terminated instantly with a warning message, thereby
detection is used to detect whether a person in front of the reducing the possibility of detecting fake faces. A block
camera is an actual person or it is his/her image or video diagram of the proposed object detection algorithm is given
displayed on a mobile device. The method inhibits Anti- in Fig. 3.
Spoofing Presentation Attack Detection, which is a major
advantage of the proposed technique. Furthermore, with the
help of a implemented Blink detection algorithm, Face
Liveness detection is ensured by eliminating the possibility
of holding an image of a person in front of the camera or any
virtual presentation. This removes one of the major
vulnerabilities of a face recognition system, which is often Fig. 3. Flow Diagram for Object Detection algorithm
noticed.
B. Blink Detection
The system model for the above-mentioned proposed In the second part of the code, the blink detection
scheme, depicted in Fig. 2 is as follows. algorithm is implemented to detect the blinks of the person
[7]. A series of steps viz. Preprocessing, Feature extraction,
Computation, Comparison of EAR and Postprocessing are
followed to compute the number of blinks.

Under preprocessing, the input from the camera is

processed and the frames are converted to greyscale via dlib.
The Regions of interest of the face are compared to the high-
resolution image that illustrates a full set of landmarks that
can also be detected via dlib library. This helps in extracting
the regions of the eyes of a person. Secondly, the facial
features are extracted from the preprocessed frames used to
Fig. 2. Proposed and Implemented Scheme calculate the Eye Aspect Ratio (EAR). This includes
extracting the coordinates of both the eyes. A high-resolution
image that illustrates a full set of landmarks that can be
The designed system is integrated with an HC-SRO4 detected via dlib library is used so that it could provide the
Ultrasonic sensor to ensure that the person is not too close to indexes to extract the eye regions of the person. Thirdly, the
the system. A distance of 20 cm is considered as an Eye Aspect Ratio (EAR) of both the eyes are computed with
appropriate distance. This in turn helps in removing the the help of equation (1) taken from [7], the coordinates used
possibility of aligning the Cell phone exactly with the frame are marked in Fig. 4.
of the camera to produce any unauthorized presentation
attempts. If any person/object is detected within the specified
distance, a warning message is displayed. The model then
proceeds with the following algorithms.

A. Object Detection
In the proposed model, object detection technique has
been used to detect the presence of any cell phone device in
front of the camera to avoid Spoofing Presentation Attack .
This algorithm uses the Common Objects in Context (COCO)
library [11], which is a pre-trained library that enables object
detection, segmentation, and captioning datasets. This library
contains thousands of images that are annotated with the
information about the objects. COCO datasets have been used Fig. 4. Coordinates of the Eye
to train a Convolutional Neural Network (CNN) to recognize
objects in the images. The CNN learns to recognize patterns
in the images that are indicative of the presence of different (1)
objects.

The code initializes a detection model using the Single It is the ratio of the Euclidean distance between the facial
Shot Detector (SSD) with a MobileNet v3 backbone. A CNN landmarks of the eye. The numerator denotes the distance
classifier used to identify and classify the images captured by between the vertical coordinates of the eye and the
the Raspberry pi camera. denominator denotes the horizontal distance.

The interesting fact about this Eye Aspect Ratio is that it

tends towards zero as the blink is taking place. Once the EAR

Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on November 22,2024 at 19:32:24 UTC from IEEE Xplore. Restrictions apply.
 is calculated, it is important to determine a threshold value so IV. IMPLEMENTATION RESULTS
as to register a blink. A threshold value is set in such a way Finally, all the above-mentioned algorithms are
that if the EAR falls below it for a specified number of frames
implemented on Rasberry pi and tested for face recognition
and then rises above, it is considered a blink. Finally, upon
detecting the specified number of blinks, in Post processing, and liveness detection, with the first author’s face as the test
the face recognition system is triggered, and the system case. The modular approach has been used to solve the
model proceeds to recognize the face of the person. overall face recognition problem.

A block diagram representing the flow of proposed blink First the object detection module as described in section
detection algorithm is given in Fig. 5. 3.1 was tested. Image of the test case taken through the
camera of a mobile phone was used for authenticating the
algorithm. The system gave the warning message ‘Mobile
Phone Detected’ as shown in Fig. 7.

Fig. 5. Block Diagram for Blink Detection Algorithm

If the blink has taken place three times, the face recognition
algorithm is triggered.

C. Face Recognition
This is the building block of the whole model. According
to the proposed method, once three blinks of the person in
front of the camera are registered, the third part of the
program i.e the face recognition algorithm is initiated. This
code identifies the person with the help of facial encodings
[12]. Fig. 7. Implementation result of object detection
A database is created of known faces by capturing images of
the individuals. This folder comprises images of the persons As a second step of the implementation, the blink
taken at different angles and are put under different folders
detection algorithm was carried out as described in section
with the person’s name as the label.
The facial features or ‘encodings’ of the captured face are 3.2. It registered the number of blinks of the person to ensure
then extracted. The encodings can be represented as a set of that there is no unauthorized access through a photograph of
numerical values that represent unique characteristics of each the person. The output of the algorithm displaying the number
face. The face recognition library used a function called of blinks of the test case is shown in Fig. 8.
‘face_encodings’ to extract the facial features. It basically
converts the image into a 128 bits vector. These encodings
are used to compare faces and determine whether they are the
same or different.

After the facial encodings have been computed, another

function, ‘compare_faces’ is used to compare these
encodings to the unknown captured face to determine
whether they match or not. If the face matches with that of
the dataset, the name of the person is displayed as shown in
the Fig. 6.

Fig. 8. Implementation result of blink detection

Finally, both the above-mentioned modules were

combined together with the face recognition model as
explained in section 3.3. It was successfully tested on the
Fig. 6. Block Diagram for Face Recognition through ‘encodings’ actual face of the first author with his name and specified
number of blinks (i.e. 3) being displayed in the command
window. The Results are shown in Fig. 9.

Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on November 22,2024 at 19:32:24 UTC from IEEE Xplore. Restrictions apply.
 ACKNOWLEDGMENT
This work was carried out as part of the internship done by
the first author at CDAC NOIDA office as part of his pre-final
year B. Tech program requirement during summer 2022. The
first author thanks the CDAC NOIDA officials for allowing to
use their lab facilities and providing all required guidance in
completion of this work.
REFERENCES
[1] Kavita Gupta, “Review paper on biometric authentication,”
International Journal of Engineering Research & Technology (IJERT),
Volume 5, Issue 23, 2018.
[2] R. Memane, P. Jadhav, J. Patil, S. Mathapati and A. Pawar,
"Attendance monitoring system using fingerprint authentication," 2022
6th International Conference on Computing, Communication, Control
and Automation (ICCUBEA), Pune, India, pp. 1-6, 2022, doi:
10.1109/ICCUBEA54992.2022.10010791.
[3] D. P. Benalcazar, J. E. Zambrano, D. Bastias, C. A. Perez and K. W.
Fig. 9. Integrated test result of face recognition Bowyer, "A 3D iris scanner from a single image using convolutional
neural networks," IEEE Access, vol. 8, pp. 98584-98599, 2020, doi:
10.1109/ACCESS.2020.2996563.
[4] Y. Kortli, M. Jridi., A. Al Falou, and M. Atri “Face recognition
V. CONCLUSION AND FUTURE WORK systems: A survey”, Sensors, Vol. 20, No.2, p. 342, 2020.
The paper envisages and proves an Anti Spoofing face [5] INC 42 report, available at “https://inc42.com/buzz/india-deploys-
facial-recognition-system-at-airports-digiyatra-beta-app-goes-lives-in-
recognition system which helps to differentiate between an delhi-bengaluru/”.
actual person and his image/video and obstructs presentation [6] Pyimagesearch blog, available at
attacks in real time applications. The model has been https://pyimagesearch.com/2019/03/11/liveness-detection-with-
implemented using an optimized framework wherein it can opencv/
be ported on any portable edge device with limited compute [7] Tereza Soukupova, Jan Cech, “Real-time eye blink detection using
capability and storage. The implemented approach also facial landmarks.,” Proceedings of 21st Computer Vision Winter
Workshop, Rimske Toplice, Slovenia, pp. 1–8, February 3–5, 2016
utilized the open source OpenCV library in conjunction with
[8] W. Wang, X. Huang, X. Shu, S. Li, H. Huang and Q. Wu “An
Object detection and Blink detection to detect Face Liveness implementation of face recognition system with face presentation
and its characteristics for application development. The attack detection on raspberry pi.” IEEE International Conference on e-
proposed model is scalable and works well on the SoCs which Business Engineering (ICEBE), pp. 70-75, 2021, doi:
offers more advantage in terms of compute and power. 10.1109/ICEBE52470.2021.00031.
As part of future work, with minor customization various [9] Vareto, Rafael Henrique, Matheus A. Diniz, and William Robson
Schwartz “Face spoofing detection on low-power devices using
applications can be developed using the existing framework embeddings with spatial and frequency-based descriptors.”
and model. The system has a huge potential of commercial Iberoamerican Congress on Pattern Recognition, pp. 187-197, doi:
deployment and can be installed in a variety of applications 10.1007/978-3-030-33904-3_17 Springer, Cham, 2019.
viz. Access Control , Security and Surveillance ,Health and [10] Dhiman, Gaurav “An innovative approach for face recognition using
raspberry Pi.” Artificial Intelligence Evolution, pp. 102-107, 2020.
Safety,Time and Attendance, eKYC and Fintech , Law
[11] COCO dataset library, available at “https://cocodataset.org/#home”.
Enforcement , Smart Retail etc.. The work is undergoing for
[12] Sharzeel Saleem, Jeba Shiney, B. Priestly Shan, and Vishesh Kumar
a System level application development using this scheme. Mishra,“Face recognition using facial features” Materials Today:
Proceedings, Vol.80, No.1, August 2021, doi:
10.1016/j.matpr.2021.07.402.

Authorized licensed use limited to: Instituto Tecnologico de Costa Rica (ITCR). Downloaded on November 22,2024 at 19:32:24 UTC from IEEE Xplore. Restrictions apply.