2023 IEEE 8th International Conference for Convergence in Technology (I2CT)

Pune, India. Apr 7-9, 2023

Biometric-based Smart Security System

Rahil Shah Shikha Punjabi Srihari Kamath
U.G. Student, Department of U.G. Student, Department of U.G. Student, Department of
Electronics and Telecommunication, Electronics and Telecommunication, Electronics and Telecommunication,
D.J. Sanghvi College of Engineering D.J. Sanghvi College of Engineering D.J. Sanghvi College of Engineering
Mumbai, India Mumbai, India Mumbai, India
[email protected] [email protected] [email protected]
2023 IEEE 8th International Conference for Convergence in Technology (I2CT) | 979-8-3503-3401-2/23/$31.00 ©2023 IEEE | DOI: 10.1109/I2CT57861.2023.10126356

Yash Dange Yukti Bandi

U.G. Student, Department of Assistant Professor, Department of
Electronics and Telecommunication, Electronics and Telecommunication,
D.J. Sanghvi College of Engineering D.J. Sanghvi College of Engineering
Mumbai, India Mumbai, India
[email protected] [email protected]

Abstract— Conventional lock-and-key methodologies have II. OBJECTIVE

a variety of inconveniences, including the inability to insert a
key into the door lock or unlock the door while one's hands are People often think innovation stems from making
full. People frequently misplace their keys. The current something completely new, innovating something from
approach to all of these difficulties is to discard physical keys scratch. However, some of the most impactful innovations of
in favor of keys that can never be lost, biometrics. This will our times have emerged out of improving and creating upon
additionally make the entire process easier, faster, and more something already existing. The same goes for our project.
convenient for an individual. Year after year, the domain of Our project makes use of existing base technologies but aims
biometric technology has advanced significantly. When to create synergies between such technologies that enable
compared with the previous technologies in the industry, each many more use cases, and an overall improvement for the
advancement attempts to promote a higher degree of security. user. The objective of the project is to create an easy-to-use,
Using the "smart system" approach, our system intends to accessible for all, secure, and robust security system that can
provide a high level of security in addition to an efficient be used in numerous situations. To do so, we aimed to train
authentication protocol. The smart system concept necessitates and create our own Machine Learning models. The models
the development of a smart, user-friendly security system that were built on custom input images, audio, and scans captured
incorporates facial, audio, and fingerprint verification as the
by us, instead of relying on a pre-existing dataset. The
three layers of authentication. It is an attempt to coalesce
fingerprint and face recognition module offers security to
existing technologies rather than developing entirely new ones.
With a few modifications in the existing system, it can be
people with speech impediments. The fingerprint and voice
employed across several applications. module provides security to people working in diverse
conditions that make it difficult to expose the face, for
Keywords—Deep Learning, Librosa, Multilayer perceptron, example, extreme cold, bio-bubbles, and more. The face and
Neural Network, NumPy, Speech Processing, Open CV, Face voice module provides security to people especially those
Recognition, Arduino UNO, Fingerprint Recognition module old, who do not have sharp fingerprints due to health
R307. conditions or aging, and those who wear gloves. This
comprehensive biometric system ensures at any given point
I. INTRODUCTION there is at least 2-factor authentication taking place. This
When one visits bank lockers or the vaults of increases the security of important objects and data while
conventional jewellers and commodity merchants, one is improving accessibility for authentic users or owners.
surprised to find that these establishments still rely on
antiquated security systems, which are notoriously III. LITERATURE REVIEW
cumbersome and time-consuming for the vast majority of The topic of security systems that use fingerprint
their customers. Not only that, but they also create high recognition has a plethora of information that may be found
barriers that inhibit people with disabilities from freely using online. The reviews shown here stand out among the rest of
the security systems without being dependent on a third them as more than a few of the others. Fernando L. Podio
person to secure their access to their valuables. Not only that, argued persuasively in his paper titled "Personal
but they also create high barriers that inhibit people with authentication through biometric technologies" that among
disabilities from using security systems. As a result, the biometric security systems such as intelligent doorknobs,
project was conceived of with these sorts of requirements in access rights, and per-view payment operations, fingerprints
mind, with the goal of implementing the most cutting-edge are one of the best options for home authentication. His
technology in very sensitive regions. Our initiative intends to paper was titled "Personal authentication through biometric
decrease the technological gaps that exist in these industries technologies." In the light of the fact that tokens such as
and give a solution that can be relied upon. We have the smart cards, magnetic stripe cards, photo ID cards, physical
technical expertise necessary to develop such complete keys, and so on are susceptible to being misplaced, stolen,
solutions. duplicated, or forgotten at home, the author asserts that
biometric security systems offer a distinct advantage over
their traditional counterparts.

979-8-3503-3401-2/23/$31.00 ©2023 IEEE 1

Authorized licensed use limited to: Shri Madhwa Vadiraja Institute of Technology & Management. Downloaded on February 24,2025 at 09:30:12 UTC from IEEE Xplore. Restrictions apply.
 It is possible for passwords to be lost, divulged, or even C. Face recognition:
just seen. However, for a wide range of applications, A collection of face encodings is presented, compared to
biometrics holds the promise of providing verification that is a known face encoding, and the Euclidean distance between
expedient, uncomplicated, accurate, dependable, and cost- each comparative face is computed.
effective. He recognized some common features that
contribute to the usability of a biometric system, such as D. Librosa:
having a distinct property, being user-friendly, having a low Librosa is a Python package that can be used to analyze
cost, being quick and easy to implement, and so on. In music and audio. It forms the foundation for developing
highlighting the economic aspect, he mentioned that systems that can retrieve information from music.
evidence of the growing acceptance of biometrics is the
availability in the marketplace of biometric-based E. SoundFile:
authentication solutions that are becoming more accurate, SoundFile is an audio library based in Python developed
less expensive, faster, and easier to use. He also mentioned on libsndfile, CFFI, and NumPy. SoundFile is capable of
that this is evidence that biometrics are becoming more interpreting and writing sound files.
widely accepted. [4]
F. Pickle:
The ability to selectively limit access to a location or
resource can be enabled through the use of a system that is A Python object structure can be serialized and
known as an access control system. A lock on a door can deserialized with the help of the pickle module, which is
serve as a physical barrier to prevent entry to certain people part of Python. Any Python object can be pickled and saved
who do not have the appropriate credentials, such as a key, to a disc using the language.
keycard, fingerprint, voice password, RFID card, security
G. Scikit-learn:
token, or coin. This can be accomplished by using a key,
keycard, fingerprint, voice password, or RFID card. The It is the most powerful and dependable ML library in
ruins of Nineveh, an ancient Assyrian metropolis, are the Python. The use of a Python interface, it provides a
location where some of the earliest known examples of key collection of efficient tools for machine learning and
and lock mechanisms were discovered [3]. Since then, statistical modelling.
technological progress has led to the development of
computers, which now provide access control options in the H. MLPClassifier:
form of downloadable software and pre-installed programs. Multi-layer Perceptron Classifier is shortened to
Voice recognition is one of the most modern ways of access MLPClassifier, and it is connected to a Neural Network.
control. This approach requires determining who is speaking MLPClassifier is a method of classification that works by
and interpreting human speech in order to function properly. employing an underlying neural network, in contrast to other
Speaker authentication refers to the process of confirming methods of classification such as Support Vectors or the
the identity of a speaker within a certain system. Enrollment, Naive Bayes Classifier.
often known as the training phase, and verification are the
two processes that make up a speech recognition system I. Adafruit Fingerprint Sensor Library:
(testing phase). During the enrollment step, the speaker's With the help of this library, you will be able to use an
voice is recorded as input signals, and then characteristics or Adafruit Fingerprint sensor with any UART, which will
qualities are extracted to form a template or model. During enable you to collect, save, retrieve, and query fingerprint
the verification process, a sample of the user's speech is data.
compared to the models that have already been saved in the
The necessary hardware components for the completion
system in order to discover the most appropriate match (es).
of this project were:
A. Arduino UNO:
IV. PROPOSED DESIGN The ATmega328P-based Arduino Uno is a
The objective of this project is accomplished by the microcontroller board (datasheet). It contains 14 digital
support of a few machine learning algorithms and libraries, a input/output pins (of which 6 are PWM outputs), 6 analogue
brief description of which has been mentioned below: inputs, a 16 MHz ceramic resonator (CSTCE16M0V53-R0),
a USB connection, a power jack, an ICSP header, and a reset
A. OpenCV: button.
OpenCV (Open-Source Computer Vision Library) is a
library of python programming functions designed to provide
a common infrastructure to solve computer vision problems
and to accelerate the use of machine perception in
commercial products.
B. NumPy:
NumPy is an important Python tool for scientific
computing. It's a Python library that includes a
multidimensional array object, as well as derived objects like
masked arrays and matrices, and several functions for
performing quick operations on arrays.

Fig. 1. Arduino UNO

2
Authorized licensed use limited to: Shri Madhwa Vadiraja Institute of Technology & Management. Downloaded on February 24,2025 at 09:30:12 UTC from IEEE Xplore. Restrictions apply.
 It includes everything necessary to support the model will be taught to recognize a user's unique speaking
microcontroller; simply connect it to a computer with a USB pattern.
wire or power it with an AC-to-DC adapter or battery to
begin.
B. Fingerprint Sensor R307:
This is the R307 Fingerprint Reader Optical Sensor
Module. R307 is a fingerprint sensor module with a TTL
UART interface, which it uses for direct connections to
microcontroller UART or PC using USB-to-serial or
MAX232 adaptor. It allows storage of data of fingerprints in
the module and configure it to function in 1:1 or 1:N mode
for identifying individuals. The module can directly interface
with 3.3v or 5v Microcontrollers. Interfacing with the PC
serial port requires a level converter (such as MAX232).
All-in-one fingerprint scanner can be included in a
variety of end products due to its integrated image collection
and algorithm chip. Users can do secondary development,
and the platform can be included in a range of final products,
including access control, attendance, safety deposit boxes,
and car door locks. Good image processing skills enable the
successful capturing of images with a maximum resolution
of 500 dpi.

Fig. 3. Flowchart of the proposed design

To increase the level of protection afforded by the

system, the user will constantly be presented with a new
phrase. The user will be able to unlock the door once they
have completed all three requirements of the lock. For the
purpose of demonstrating how to open the door, the lock
would be created with the use of a screw and a motor driver.

Fig. 2. Fingerprint Sensor Module R307

Fig. 4. Circuit Diagram of Fingerprint Analysis Section

V. IMPLEMENTATION
In our biometrically based smart security system, we use A. Fingerprint Recognition:
a three-factor authentication mechanism for a greater sense In order for the Arduino to function, an external supply
of security. Therefore, the project is comprised of three of 5V DC must be given. A jumper wire is used to link the
distinct elements. There is a component for facial ground pin of the R307 module to the ground pin of the
recognition, in which a web camera is used to take a picture microcontroller in order to complete the connection. The
of the user and then save it in the system's database. A UART pins of the fingerprint sensor, specifically Tx and Rx,
machine learning model developed using OpenCV and face are utilized in order to facilitate asynchronous serial
recognition libraries will be used in order to detect and communication between the two devices. On an Arduino, the
identify the user's face each and every time the user attempts digital pin labelled D2 is connected to the transmit (Tx) pin,
to log in. while the digital pin labelled D3 is attached to the receive
The second component is the recognition of the user's (Rx) pin. Jumper wires are utilized in the process of
fingerprints. A user's fingerprint is able to be detected and establishing the connections. There is a connection made
saved in the database thanks to the fingerprint sensor module between the VCC pin of the fingerprint sensor and the 5V
R307’s assistance in doing so. pin of the microcontroller.
The third component is speech recognition, in which a After ensuring that all of the necessary connections have
CNN-based deep learning model, is trained using a dataset of been made, it is now time to enroll a finger in order to
20 distinct statements for a certain individual who wishes to construct a database. Once the 'enroll' code from the Adafruit
pass the security check. Multilayer Perceptron Classifier or Finger Print Sensor library has been executed, we are able to
MLP Classifier is being used to execute the model. The assign a number in the range of 1 to 127 to the R307 sensor

3
Authorized licensed use limited to: Shri Madhwa Vadiraja Institute of Technology & Management. Downloaded on February 24,2025 at 09:30:12 UTC from IEEE Xplore. Restrictions apply.
 so that it may store a fingerprint's data. Following the
selection of a particular id to be associated with the
fingerprint, the user will be required to insert his finger twice
for the data to be saved. This is the process by which data is
saved.

Fig. 6 (b) Same Audio with sound virtually eliminated

Although it boosts the performance of models

significantly all by itself, noise reduction isn’t the only pre-
processing we would have to do in order to build a
functioning model. Various other parameters of sound, such
as Bit Rate, Sample Rate, Number of Channels (mono,
stereo), and compression technique, must all be uniform for
all the audio files used for training as well as testing
purposes. The two screenshots below provide a detailed
description of a sound file before processing and posting it.
The software used for this purpose is Adobe Premiere Pro,
Fig. 5. Fingerprint Enroll Phase and Audacity.
B. Speech Recognition and Speaker Identification:
A significant part of the information provided by audio is
increasing the amount of digital content now available,
which has led to a requirement for procedures that
automatically evaluate such content. One of the key areas of
speaker identification is study conducted using voice signals.
Other prominent areas of it are Speech to Text Conversion,
Speech Recognition, and vice versa also, etc. the MFCC, or
Mel Frequency Cepstral Coefficient a crucial element in the
Speaker Identification process. However, as an alternative to,
there are various feature lists accessible. Linear Predictor
Coefficient (LPC), Spectrum, and MFCC Line, Rhythm,
Sub-band Centroid (SSC), and Turbulence ChromaFactor,
Spectral Frequency (LPF), etc. For the purpose of this
Fig. 7. M4A Sound file Properties (before processing)
project, we have defined a function called ‘extract_features’
which converts the given waveform input file on the ‘mel’
scale and is used to derive MFCC, and ChromaFactor,
stacked on top of each other horizontally. This gives rise to a
numeric array, which is fit for model training.
The most vital step of the training process was creating a
custom database of unique sentences recorded by the group
members. Initially, a database was created of members
speaking numbers, and singular words. However, the model
was not trained well enough. Hence, the database was later Fig. 8. WAV Sound file Properties (after processing)
shifted to contain entire sentences, 20 per person to be
precise. 15 used for training, 5 for testing. Each voice sample After performing the same ‘cleaning’ process, the files
was stored and inputted in a .wav format at a frame rate of are now ready to be stored as a database and can be used for
44.1kHz, as a mono channel input. model testing. A sample waveform distribution of a cleaned
However, every recorded piece of audio cannot be used file can be shown below:
directly, as the presence of white noise, background
disturbance and distortion massively impact the performance
of the model. As an example, the figure below shows the
waveform depiction of an audio sample, before and after the
noise has been removed.

Fig. 6. (a) Audio Sample with the presence of noise Fig. 9. Final Input Waveform

4
Authorized licensed use limited to: Shri Madhwa Vadiraja Institute of Technology & Management. Downloaded on February 24,2025 at 09:30:12 UTC from IEEE Xplore. Restrictions apply.
 C. Face Recognition: B. Speech Recognition & Speaker Identification:
To create a face recognition system, the help of OpenCV Using the Multi-layer Perceptron, we achieved an
and face recognition libraries were taken. A database was accuracy of 100% on the test set, as the model accurately
created containing, labeled photos of each person, making identified all 20 speakers (4 people, 5 test samples each).
sure the face was well lit. Then the face recognition library The screenshot demonstrating an example is shown below:
matched the input unlabeled photo with the labeled input
photo, and searched for the nearest match, i.e., the photo with
the least distance. The face was highlighted by creating a box
around it. The program was then modified to work on live
input stream from the on-device camera and was able to
detect and identify people accurately and mark people not a
part of the database as unknown. Fig. 11. Accurate Identification of speaker – Rahil (Test Set)

VI. SOCIAL IMPACT Despite the almost perfect accuracy, challenges still
One of the biggest social impacts of the project is the persist when trying to implement the same in a ‘real-time’
improved accessibility to physically disabled people. setting. This is because the same amount of pre-processing
Physical disability lies on a spectrum and our project (using professional mechanisms like Audacity and Adobe
addresses as many conditions as possible. If a person has Premier Pro) can’t be performed on the received audio
undergone facial surgery, be it due to an accident or cosmetic samples, and we will have to rely on Python and its libraries
purposes, it ensures the rightful owner can still easily access to natively clean the files. This increases the possibility of a
their property. Another impact is the saving of time spent by misjudged classification, but there are measures that can be
bank staff, to physically escort and help unlock safes. taken to reduce the chances of that happening. Firstly, using
Remote biometric authentication capabilities mean, the staff a premium quality sensor microphone, with additional tools
can securely perform authentication from their desk, for eliminating background sounds (such as a dead cat/anti-
reducing queues in banks and saving valuable time. wind foam cap) can massively elevate the quality of sound
input given to the model, and thereby boost chances of a
VII. RESULTS AND CONCLUSION better prediction. In addition, various python libraries can be
used to further enhance the standard of sound fed to the
A. Fingerprint Recognition: model, which will ultimately yield better results. As an
Once the 'fingerprint sensor’ code from the Adafruit example of a real-time prediction, an instance has been
Fingerprint Sensor library has been executed, the R307 shown below where the model accurately predicted the
module constantly flickers and continuously tries to look for speaker.
a fingerprint. Once the user places the finger on the sensor,
the sensor will try to match the instantaneous image captured
with its current existing database.
If the image successfully matches with a high value of
confidence, it displays success and authenticates the user or
else it displays fingerprint did not match and restricts the
user. For the extent of this project, a total of 4 different
fingerprints were stored in the database, and the sensor was
able to detect each of them correctly on all occasions, and
also appropriately displayed error messages for unregistered
Fig. 12. Accurate Identification of speaker – Yash (Real Time)
fingerprints. The module R307 can store a total of 1000
fingerprints, which well exceeded the project’s requirements. As per our observations, the performance of the model
depends massively on the microphone used and the nature of
the environment. For code ‘0’ or the built-in microphone in
the laptop, poor predictions are obtained since the sound
quality deteriorates immensely. When we use the same
phone (Samsung Galaxy S20 FE) as used to create the
database, estimates are more consistent. As a metric, the
model gets accurate predictions 70% of the time, in the 40-50
instances we tested it in.
To overcome this issue, a virtual cable application was
used, that when installed created a virtual link such that it
transmitted the audio recorded by the mic on one device to
the other, as its audio input. This enabled the usage of the
high-quality microphone of a smartphone to be used to take a
live recording and input it to the PC. The PC gives a live
instant output of the voice, with a high accuracy.
C. Face Recognition:
Fig. 10. Fingerprint Recognition Phase Below is the output detecting a random unlabeled photo,
via a live stream captured by the device camera. The same

5
Authorized licensed use limited to: Shri Madhwa Vadiraja Institute of Technology & Management. Downloaded on February 24,2025 at 09:30:12 UTC from IEEE Xplore. Restrictions apply.
 person, Jeff Bezos, was registered in the model with a advanced and safe method of identification that is generally
different unlabeled photo. The given model also worked on immune to any known kind of impersonation. Gait tracking
team members, accurately identifying and differentiating and detection are also known as biometric gait analysis. A
between them. The model performed adequately on most powerful method of authentication that can be utilized by
occasions and was able to easily identify and distinguish financial institutions and other industries that place a high
between different members in the group. It was also able to priority on client safety and security is a gait identification
identify 3-4 people at once. However, the program faced system that has been adequately educated and is equipped
difficulties identifying and labelling individuals in dark with the requisite hardware.
lighting conditions, owing to the limitations of the webcam
used. Similar to the speaker identification system, this can be Finally, one of the most essential upgrades to the system
addressed, and the performance can be significantly will be the introduction of chip-level encryption, which
improved by the usage of better-quality web cameras, stores data in an encrypted format, such that even if the
preferably with a flashlight. system were to be physically compromised and accessed, it
would still not reveal any sensitive biometrics stored to any
malicious entity.
ACKNOWLEDGMENT
This initiative and technical report received strong
backing from Mrs. Yukti Bandi. Her comprehensive
guidance and proper mentoring at every stage from writing
this paper to valuable insights was helpful. We appreciate
our colleagues who offered advice and knowledge that made
significant contributions to the research, and supported us
wholeheartedly throughout the whole procedure.
REFERENCES
[1] JigmeYeshi, Kazuhiro Muramatsu, “Dual Door Lock System Using
Radio-Frequency Identification and Fingerprint Recognition”, 2019.
[2] HashemAlnabhi, Yahya Al- Naaman, Mohammed Al- madhehagi,
Mohammed Alhamzi, “Enhanced Security Methods of Door Locking
Fig. 13. Face Recognition Output Based Fingerprint”, 2020.
[3] Rohit Kumar Dubey, Jonathan Goh and Vrizlyn L, “Fingerprint
VIII.FUTURE SCOPE Liveness Detection From Single Image Using Low-Level Features
and Shape Analysis”, 2016
The product in its current iteration is dependent on a [4] VikasGoyal, Himanshu Jindal, “Improved fingerprint matching
number of different platforms in order to operate efficiently minutiae singular points network”, 2017
and safely. The subsequent step would be to develop a single [5] PravjotKaur, Gagandeep Jagdev, “Reconnoitering and Instigating
platform that is capable of natively running all three types of Fingerprints as Secure Biometric Technique”, 2017
authentications while also incorporating all three into a [6] Wu Ping, Wu Guichu, XieWebin, Lu Jianguo, Li Peng, “Remote
single system. This would lower the cost, as well as the Monitoring Intelligent System Based on Fingerprint Door Lock”,
number of dependencies, that are required to execute all 2010
three modules. In the longer term, it would make the entire [7] Gudavalli, Monica, Vidyasagar Novel, “Door Lock Operation Using
project commercially viable because it would become a plug- Two-Staged Smart Security Verification”, 2019
and-play product for the customer, who would not have to [8] Anu and Bhatia, “A smart door access system using fingerprint
biometric system”, 2014
fuss over the installation and running of the system. This
[9] Hema Lathaatha S, “A systematic review on Fingerprint based
would eliminate a significant barrier to entry for potential Biometric Authentication Systems”, 2020
buyers of the product. A fourth module will be added for
[10] 10. OnyanA, Enalume K, “Property Security Using a Biometric Based
ultra-secure situations to improve the level of security Door Lock System”, 2018
already present. Tracking and detecting gait is a highly

6
Authorized licensed use limited to: Shri Madhwa Vadiraja Institute of Technology & Management. Downloaded on February 24,2025 at 09:30:12 UTC from IEEE Xplore. Restrictions apply.