Project Title: FACE RECOGNITION WITH MASK

Project Guide Name: Dr. Virender Kumar Mehla

Designation: ASSOCIATE PROFESSOR
Department: SOEIT
Contact Number:7015865834
Email ID: [email protected] Sign of Guide.

Name: PRAJWAL PANDEY

Enrollment No : 2302209014
Contact Number: 9520171219
Email ID: [email protected]
Abstract

Face recognition has become a vital biometric authentication technique, widely used in
security, surveillance, and personal authentication. However, with the widespread use of face
masks due to health concerns like COVID-19, traditional face recognition systems have faced
challenges in accurate identification. This project aims to develop a face recognition system
that can accurately identify individuals even when they are wearing masks. The system will
leverage deep learning techniques, specifically convolutional neural networks (CNNs), along
with machine learning classifiers to improve recognition accuracy. The proposed model will
be trained on masked and unmasked face datasets to enhance robustness. The implementation
will be done using OpenCV, TensorFlow/Keras, and Python, with real-time detection
capabilities.

Literature Survey
Overview

[2020]

Paper (1) The Effect of Wearing a Mask on Face Recognition Performance: An Exploratory
Study by Naser Damer et al. (2020)

Summary: This study investigates how face masks impact the performance of automatic face
recognition systems. The authors conducted experiments to evaluate recognition accuracy
with masked faces, providing early insights into the challenges posed by face coverings.

Paper (2) NIST Finds Flaws in Facial Checks on People with COVID Masks by L. Lane
(2020)

Summary: This report from the National Institute of Standards and Technology (NIST)
examines the performance degradation in facial recognition systems when subjects wear
masks. It highlights the need for improved algorithms to handle masked face recognition.

[2021]

Paper (1) Review on Literature Survey of Human Recognition with Face Mask by Dr.
Vandana S. Bhat et al. (2021)

Summary: This paper provides a comprehensive review of various methods and algorithms
used for human recognition with face masks. It discusses approaches like Haar cascade,
Adaboost, and VGG-16 CNN models, offering a comparative analysis to determine feasible
solutions for masked face recognition.

Paper (2) Masked Face Recognition: Human vs. Machine by Naser Damer et al. (2021)

Summary: This work compares the face verification performance of human experts and
state-of-the-art automatic face recognition solutions when dealing with masked faces. The
study provides insights into the challenges and effectiveness of both human and machine
recognition in the context of face masks.
[2023]

Paper (1) Masked Face Identification and Tracking Using Deep Learning: A Review by
Shahad Fadhil Abbas et al. (2023)

Summary: This review article discusses the challenges and advancements in masked face
identification and tracking using deep learning techniques. It proposes the Masked Face
Identification and Tracking (MFIT) model, incorporating architectures like YOLOv5 and
attention mechanisms, and evaluates standard datasets relevant to face mask detection and
tracking.

Paper (2)"An Exploratory Study of Masked Face Recognition with Machine Learning
Algorithms" by Megh Pudyel and Mustafa Atay (2023)

Summary: This study evaluates the performance of various machine learning algorithms in
recognizing masked faces. Using Local Binary Pattern (LBP) for feature extraction, the
authors assess six conventional algorithms and explore different training scenarios, including
unmasked, masked, and half-masked datasets, to understand their effectiveness in masked
face recognition.

[2024]

Paper (1) Masked Face Recognition with Generative-to-Discriminative Representations" by

Shiming Ge et al. (2024)

Summary: This paper proposes a unified deep network that learns generative-to-
discriminative representations to enhance masked face recognition. The approach involves a
generative encoder for face inpainting and a discriminative reformer, effectively addressing
the challenges posed by diverse occlusions in masked faces.

Paper (2)-"Artificial Intelligence-Based Masked Face Detection: A Survey" by [Authors Not

Specified] (2024)

Summary: This comprehensive survey analyzes masked face detection methods utilizing
artificial intelligence. It examines various techniques, datasets, and challenges associated
with detecting masked faces, providing a thorough evaluation of the performance and
accuracy of different approaches in the context of the COVID-19 pandemic.

Related Work

How many organizations rely on image processing and object recognition to identify security
issues and image classification, notably in health image processing .The prime objective for
this research is to create a face recognition algorithm that combines real-time computer vision
(CV2) with CNN (an advanced learning algorithm) to recognition a person who is wearing a
mask .Figure 1.1 shows that the classification results are being shown independent of whether
the individual is wearing a mask or not.
 Figure 1.1 Image Classification Steps

The proposed model’s main principle is that in the first phase, a random image dataset that is
easy to access online is collected and processed even-before being uploaded to the server.
Following the generation of the dataset, the next step in this field is to include image pre-
processing to recover elements and improve the model’s accuracy. Subsequent to
preprocessing, the model must be trained to predict the final output, which can take some
time dependent on the hardware setup. The second step is to recognize real-time computer
vision data using an CV2-built library and, finally, to present the results to determine whether
or not a person is wearing a mask.

IMPLEMENTATION AND METHADOLOGY

How many organizations rely on image processing and object recognition to identify security
issues and image classification, notably in health image processing. The prime objective for
this research is to create a face recognition algorithm that combines real-time computer vision
(OpenCV) with CNN (an advanced learning algorithm) to recognise a person who is wearing
a mask . The proposed model’s main principle is that in the first phase, a random image
dataset that is easy to access online is collected and processed even-before being uploaded to
the server. Following the generation of the dataset, the next step in this field is to include
image preprocessing to recover elements and improve the model’s accuracy. Subsequent to
preprocessing, the model must be trained to predict the final output, which can take some
time dependent on the hardware setup. The second step is to recognise real-time computer
vision data using an OpenCV-built library and, finally, to present the results to determine
whether or not a person is wearing a mask, Image processing and data augmentation Pre-
processing data assists in the cleaning of data, the extraction of important characteristics from
a dataset, and the improvement of model performance in classification. With an Figure 4.2:
Analysis and Classification of Image augmentation algorithm, image pre-processing helps in
transmitting information, improving images, and making new data with smaller image
datasets .The size, contrast, position, and orientation of a classified data obtained may vary.
Image pre-processing is therefore the systematic, algorithm-based step done to ensure that
images are correctly edited and ready for training and inference. The image dataset is pre-
processed using augmentation methods. It may randomly crop, color filter, position, flip, and
add noise to the training data using Image Generator and OpenCV built-in functions to
minimize model overfitting. This study uses other pre-processing methods and augmentation
to create a verified dataset of various types. the system will send preprocessed images to any
advanced learning algorithms to extract features. Several data augmentation methods are used
to ensure that the dataset increases in size and quality. This assisted in reducing overfitting
issues and improving the model’s overview ability during training.

image augmentation settings that were employed. So, first and primarily, the data is resized
by a value of 1/ 255. This aids in the normalization process. Following that, a zoom range of
20% is used to allow for some random cropping of the image. The rotation range is set to 40
degrees, and the horizontal flip is enabled. 27 4.2 CNN Model CNN was a creator in image
processing tasks including text classification and object recognition. When it comes to
collecting features from a dataset for image classification or object recognition, the
Convolutional Neural Network technique uses other advanced learning algorithms.
Convolutional filters and the ReLu activation function remove items from the Convolutional
Layer. Convolutional neural networks (Conv2D) have been frequently used for image
classification, however based on this concept, only one dimension has been used for Table
ttings Methods Setting Rescale 1/255 Zoom Range 0.2 Rotation-Image 40 Horizontal Flip
True text classification. This model was created for human identification in the presence or
absence of a face mask, and it has been modified to match the challenges in the domain. The
major parts of this Convolutional Neural Network model are given here. 4.3 Data image To
ensure that the model runs efficiently, the collected input image is set for pre-processing.
Data preprocessing techniques are used to improve datasets. Due to the worldwide pandemic
of COVID19, physical data collection is complicated. To extract major elements, the
enhanced data should be sent to the preceding convolution 4.4 Stages in Convolution
Convolutions extract characteristics from large sets of input data and multiply the generated
N N matrices. For image classification, Conv2D uses filters, kernel size, input shapes, and
activation functions. The values of the variables in this research area are already listed in
table 4.1. 28 4.5 Pooling Layer Maximum Max-pooling reduces dimensionality and extracts
the most features in advanced learning algorithms. By calculating the average of the provided
elements, the pooling layer reduces the number of variables and regularizes Layer fully
connected It draws pooling layers, straightens them, and transfers the others into the next
layer. In CNN, the fully connected layer is essential for classification. The following
categorization result is achieved with the help of the activation function .The proposed
network architecture’s layers are described Generate_database.pyThis python file is used to
generate the mask on each image and then collect them. The Python libraries used for image
recognition and mask generation are tqdm, numpy, cv2. The mask generated will be like the
image Generated Dataset Examples The database will be generated according to the height,
length, and breadth of the image and accordingly it will resize the image in a suitable pixel.

Steps to make a face recognition mask detection system using Python and deep
learning:

1. Collect Dataset – Gather images of masked and unmasked faces.

2. Preprocess Data – Perform image augmentation (resizing, filtering, cropping).
3. Build CNN Model – Use Convolutional Neural Networks (CNN) for feature
extraction.
4. Train Model – Train the model using labeled datasets with deep learning
frameworks like TensorFlow/Keras.
5. Validate Model – Evaluate accuracy using test datasets.
6. Implement Real-Time Detection – Use OpenCV to capture live video input and
detect masks in real time.
7. Deploy System – Integrate the model with a user interface or security system for
practical application.
Existing System

 Traditional face recognition systems rely on full-face visibility for accurate

identification.
 Common algorithms include Eigenfaces, Fisherfaces, and Local Binary Patterns
Histogram (LBPH).
 These methods struggle with occluded faces, such as those covered by masks, leading
to high error rates.

Proposed System

Tools / Hardware / Software Required

Software:

 Python
 OpenCV
 TensorFlow / Keras
 NumPy, Pandas, Matplotlib
 Dlib (for face detection)

Hardware:

 Laptop/PC with GPU (NVIDIA recommended)

 Camera module (for real-time testing)
Base Paper Details

 Title: "Masked Face Recognition using Deep Learning and CNNs"

 Authors: Amit Kumar Mishra, Geetansh Verma, Aarushi Singh, Vishal Kumar,
Manoj Diwakar, and Prabhishek Singh.
 Published in: IEEE / Springer / Elsevier
 Year: [2024]
 Summary: This paper discusses a CNN-based approach to improve face recognition
accuracy when individuals wear masks. It presents a modified architecture and
evaluates its performance on benchmark datasets.

References
[1] Damer, N., Grebe, J. H., Chen, C., Boutros, F., & Kuijper, A. (2020). The Effect of Wearing a
Mask on Face Recognition Performance: An Exploratory Study. arXiv preprint arXiv:2007.14383.

[2] Lane, L. (2020). NIST Finds Flaws in Facial Checks on People with COVID Masks. National
Institute of Standards and Technology (NIST).

[3] Bhat, V. S., Sonawane, P., & Gite, S. (2021). Review on Literature Survey of Human Recognition
with Face Mask. International Journal of Engineering Research & Technology (IJERT

[4] Damer, N., Grebe, J. H., Boutros, F., Kirchbuchner, F., & Kuijper, A. (2021). Masked Face
Recognition: Human vs. Machine. arXiv preprint arXiv:2103.01929.

[5] Abbas, S. F., Ahmed, A. M., & Salih, H. (2023). Masked Face Identification and Tracking Using
Deep Learning: A Review. Journal of Computer Science, 19(10), 1423-1437. DOI:
10.3844/jcssp.2023.1423.1437.

[6] Pudyel, M., & Atay, M. (2023). An Exploratory Study of Masked Face Recognition with
Machine Learning Algorithms. arXiv preprint arXiv:2306.08549.

[7] Ge, S., Wang, J., & Yang, J. (2024). Masked Face Recognition with Generative-to-Discriminative
Representations. Proceedings of Machine Learning Research, 235.

[8] [Authors Not Specified] (2024). Artificial Intelligence-Based Masked Face Detection: A Survey.
Science Direct. DOI: 10.1016/j.patcog.2024.107536.