Volume 6, Issue 2, February – 2021 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Facial Recognition Using Deep Learning

Ayush Bijouraa , Sritama Banerjeea , Kirti Sekhar Pandeya , Dr Arivoli Ab
a b
Mtech, Dept. of Computer Science, Vellore Institute Of Technology, Vellore, TamilNadu, India. Assistant Professor(Sr) School
of Computer Science and Engineering, Vellore Institute Of Technology, Vellore, TamillNadu, India.

Abstract:- Facial expressions does not involve information available in the database, all these parameters matches, then
between humans in face to face interactions. Facial the person is identified.
recognition is automatic, and has a important role in
human-machine interfaces, this has attracted increasing The approach towards machine learning will often
the attention of researchers from nineties. The approach require a complex feature extraction process and produce poor
towards machine learning always require a feature results. In this project data will be stored in form of image. It
extraction process and always produces poor results. In will be stored as csv file .We will divide the data into two sets
this paper, we will use real time data set and compare it i.e. training dataset and testing dataset. After processing the
with the current image .We will also detect the image from image will go for thresholding, segmentation, canny edge
the running video then we will use various detection detection, camera detection, ROI (Region of Image) and Hard
techniques to detect the image. In this way we will come to Cascading. Then we will keep the image in Open CV library.
know whether the person is authorized or not. We will also First it will detect the image and it will find whether it is the
find the current mood of the person from the detected image of a human being or not. If it is human being, it will
image. return “Image Detected”.

Keywords:- Deep learning, Facial Expression Recognition, After all these processing the image will get converted in
Neural Network. the form of pixels and it will compare the current image with
the image or video stored in the database as dataset[5]. If it
I. INTRODUCTION matches we will be able to find that the person is a member of
particular organization. In this way it will be used for security
Face recognition[1] can be defined as the process of purpose as we can find whether the person is authorized or
identifying and verifying people(s) in the photograph by their not.
face. The process comprises of detection, alignment, feature
extraction, and a recognition task. This is a computer vision The new concept in our project is that it will detect the
program which helps in finding the faces in photo(s). image from the current running video also. The image which
is detected we will find the current mood of the person.
Deep Learning[2]can be defined as a worldview of
Machine learning, or all the more exactly one of its II. RELATED WORK
calculations. For the best degree, it depends on an idea of a
human cerebrum and the connection of neurons. Deep learning After processing, the image will go for thresholding[6],
has the ability to construct recognition[3] biometric software segmentation, Canny edge detection, camera detection, ROI
which is capable of confirming an individual. The (Region of Image) and Hard Cascading.
characteristics that seem common to us in humans (example,
 OpenCV
color of eye) which will not make any sense for a computer
OpenCV short for Open Source Computer Vision can be
which is analyzing each and every pixels in an image. The
defined as a library of programming functions and are mostly
most important thing while recognizing the face is the how
used in real-time computer vision. Basically, it is library
much is the distance between the eyes, and what is the width
which is used for processing the image. It is mostly used to do
and length of the nose, chin width , other things. After this, the
all the operation(s which are related to image(s).
data which is obtained are compared with those data which are

IJISRT21FEB446 www.ijisrt.com 512

Volume 6, Issue 2, February – 2021 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165

Fig 2.1:- Representation of Deep Neural Network

This process can also be done by Keras. Keras are used a variety of situations.Edge Acquisition is a low cost, which
in deep learning to classify the images and to build a means that the acquisition should take exactly as many edges
convolutional neural network (CNN). shown in the image as possible. The point received by the
operator should apply directly to the center edge.
The images are seen by computer using pixels. Pixels are
related to images. For an example, group of pixels may mark The edges provided in the image should only be marked,
an edge in an image or it may define some other pattern. and where possible, the sound of the image should not create
Convolutions are used and help to identify the images. A false edges. To satisfy these needs Canny used a different
convolution multiplies a matrix consisting of pixels with a calculation - a technique that gets the job done right. The
filter matrix and then sums up the values of the multiplication. proper function in the Canny detector is defined by the sum of
The convolution then slides to the next pixel and same process the four exponential terms, but can be estimated by the first
is repeated till all the pixels in the images are covered. Gaussian output.

 Image Thresholding: Among the most advanced edge detection methods to

Image threshing is very simple, and effective way to date, the Canny detection algorithm is one of the most
separate an image into a frame and a layer. It is a type of described methods that provides good and reliable detection.
image classification which separates the objects by converting Due to its combination of three-dimensional detection and
the grayscale image(s) into binary image(s). To process an simplifying the implementation process, it became one of the
image, the threshold is used to divide the image into smaller most popular algorithms to find.
sections using one color or grey color at least to define the
border. The main advantage of getting a binary image first is  Camera detection
that it reduces complexity and simplifies the process of Most home security cameras[7] these days have facial
recognition and classification recognition, allowing you to build a database of friends and
family who visit your house frequently. After that, when the
 Image Segmentation camera detects a face, it decides whether or not your known
Image segmentation[8] (in terms of computer vision)can database is a name.
be defined as the process of dividing a digital image into
different categories (pixel sets, also known as photo objects). The chip inside the camera is constantly scanning the
The main purpose of segmentation is to simplify a or to image on its burner, nose, ears and teeth, making it 10 faces at
transform the representation of an image into a logical and a time before hitting the shutter. Generally, scanned faces
easy-to-diagnose component .A part of the art section is a way should add at least 10 percent of the LCD view height - this
of dividing or dividing an image into parts, called parts. It is requirement prevents the camera from trying to lock face-to-
very useful for applications such as image reduction or object face in the background. You can turn off the facial features
recognition, because in these types of applications, it is not when your subject is not personal.
efficient to process the entire image.
 ROI (Region of Image)
 Canny edge Detection Performance is measured according to the accuracy of
Canny edge detection[10] is a multi-step algorithm that the facial recognition category, as well as the accuracy of ROI
can detect edges with high noise simultaneously. Slide images detection[9] from various facial images. To evaluate
with Gaussian filter to reduce unwanted noise and details. performance, the proposed face algorithm was evaluated by
Canny found that the application requirements for finding assigning the face value N and the number of Negative Faces
edges in different viewing systems were the same. Therefore, M.
a solution to find an edge to address these needs may work in

IJISRT21FEB446 www.ijisrt.com 513

Volume 6, Issue 2, February – 2021 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
 Neural Networks
Neural networks can be used to recognize the face(s)by
learning the coefficients and the calculation is done by the
Eigenfaces algorithm. After this the network is then trained on
the images present in the database, and then ready to identify
the face images which are given to it. The principal
component analysis is used to reduce the input series of the
neural network, so that the running time of the application can
be reduced. The parameters are shown in Fig. 1.2 and is
calculated by the formula Input unit(IU)= number of
picture/person.Output unit(OU)=total no of person(s) which is
to be recognized.

Fig 2.2 Region of Image

 Hard Cascading
Skipping Cascading is a case study[11] based on a wide
variety of students, using all the information gathered from
the findings from the classifier provided as additional details
for the next lesson at Cascade. Unlike volumes or ensembles,
which are many expert systems, skiing is a multistage one.

Cascading separators are trained by looking at large

"good" samples of something and counterfeit "not good"
images of the same value. After the student is educated he can
be placed in the photographic region and see the object being
discovered. To search for an item in the entire frame, a search Fig 3.2 :- Network Architecture
window can be moved across the image and scanned wherever
the subject is placed. This method is widely used in image  System Architecture
processing to locate objects and track them, especially for
visual and facial recognition.  Overview of EigenFace
Eigenfaces are defined as a related set of facial
III. METHODOLOGY characteristics that a computer uses to recognize a person's
face. In this technique, we are having training and test set of
Face recognition is where images from the video and images, and then we will compute the eigenvectors of the
still pictures are emerging as a research area having different covariance matrix of the training set of images. They do not
applications in various other fields like law, commercial, etc. directly correspond to any of the features of face like eyes,
This system would find numerous applications like criminal nose and mouth. When the eigenvectors are displayed, they
identification, finding missing persons, credit/debit card look like a ghostly face, and they are termed eigenfaces.
verification and many more.
 EigenFace Representation
When the system[12] is designed ,different problems are To create an eigenface, a set of images[13] which are
needed to be addressed. For example when image is detected large are collected. These are the images which are the good
the pattern of image are taken as a subject and then against representation of all faces that one might occur. When we
either a uniform or a complex background. Then, the configure the first set of the images of face .These face
identification and verification of face images are done using a image(s) are converted into row vector by scanning each of
proper classification algorithm and then the processed results these face(s) image from left to right, top to bottom. And after
are posted using schemes which is based on model. all this all the face vectors are combined so that each row must
have a face vector and the whole face distribution is made.
Now , here the training set will have N images, the principal
component analysis will form the basis set images set M, here,
M < N. The reconstruction error can be reduced by increasing
the eigen pictures numbers , N is always chosen less . For
example, if we want to generate M a number of eigenfaces for
Fig 3.1:-Face detection process a given training set for N face images, then we can say that
each face image is to be made up of proportion(s) having all

IJISRT21FEB446 www.ijisrt.com 514

Volume 6, Issue 2, February – 2021 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
the K features or the features of eigenfaces: Face image = which are present on the 2D face images they are detected
(24% of E1) + (3% of E2) + (52% of E3) + ... + (2% En). using the conventional Active Appearance Model (AAM).

 Training Data  3D Aging Model

While starting the training , here we will select only ten The aging model can be defined as the aging pattern[16]
face image(s) of individual person which are properly aligned which comprises of face array model from a single subject
and having a frontal view. The extraction of face extraction is which is indexed by the given age of the person. So, by
done by the face extraction unit[14]. assuming that any aging pattern will be approximated by
average weighted which is having aging patterns present in the
 Training the Network training set. Here, the model construction will differ mostly,
We have to prepare a set of training data, for this we take here the modeling shape and texture differ separately at
ten images of the same person with varying face different ages using the shape which is pattern space in aging
expressions.For training to get started we load all the training and the texture which is an aging pattern space, because of the
face picture to the network, here not whole picture is loaded to fact that the 3D shape and the texture images they are less
the network but the projection of the picture using the correlated than the 2D shape and texture which is in use. Here,
eigenface space. we will also adjust the shape of 3D which is explained below.
Here, the different pattern spaces are described below.
 Classification
Classification can be defined as a process of categorizing  Shape Aging Pattern
the classes in the given set of data. This classification can be The Shape pattern space[17] will capture the variations
performed on both the structured and unstructured data. When in the the shape and the size of the face which is internal.
the data is trained, the face recognizer will be ready for These can be obtained from preprocessing phase and will be
recognition. used for construction the shape of the pattern space.

 Age Recognition Those people who are under the age of 18, the key
Age[14] can also be recognized from face. Since, aging effects of aging are seen in the increase in the size of carnial,
is a complex process which will affect the shape, size and in the later ages, the facial growth[18] is very small
texture of the face. The aging process is also visible in considering the height and width. Those who are under the age
different age groups. The aging in the face is represented by of 19, we are rescaling the overall size and shapes of 3D. We
growth in the face of age groups (for example, the face of 18 will perform PCA over all the 3D shapes, where, Sji, is
years old), is represented by major texture changes and small irrespective of age j and is subject to i. All the mean is
changes in the shape. These are due to the changes in the subtracted Sji into the subspace which is spanned by the
weight and thickness of the skin, people whose age groups are columns of Vs to obtain Sj i.
greater than 18 years. Therefore, the age adjustment is must
which can compensate both types of aging processes.

 2D Facial Feature Point Detection (1)

Here, we are using manually marked feature points in It is an Ls * l vector.

aging[15] model construction. Therefore , in the test stage, the
feature points are detected automatically. The feature points

Fig 3.3:- Simulation of age which is ranging from x to y.

IJISRT21FEB446 www.ijisrt.com 515

Volume 6, Issue 2, February – 2021 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
 Texture Aging Pattern given j as an age and a set of weights wt, as shown below.
Here Tj is the texture pattern where i is subject to age j
and j can be obtained by putting the original image of the face
to frontal projection which is in the mean shape of S which is
(2)
followed by concatenating the pixels of image in column wise.
After applying PCA on Tji , we will calculate the Vt which is
transformation matrix and the texture which is projected t, j, i.
We will follow the same procedure in filling which is as in
(3)
the pattern shape space which will be used to construct the
complete basis for the pattern space shape by using t j i. So,
the new texture which is Tj can be obtained in the same way,

Fig.3.4 (a)At age 2 the image is given. (b) From age 0 to 60 the simulated images are shown.(c) Five different poses from the aging 20
is. (d) Original images

 Emotion Recognition IV. CONCLUSION

Emotion[19] of a person can be recognized using his/her
facial expressions. Based on his/her facial expression[20] By working on this project we found that when camera is
whether the person is sad, happy, angry ,neutral , joyful. The capturing the face ,it is displaying the mood of person whether
structure is shown below. he/she is happy, sad, angry, neutral ,joyful . We have also used
Open CV and CNN which is working properly and the
algorithm which we have used has shown efficiency. The
proposed model is able to recognize faces effectively but when
experimented with videos, it takes more time to process. The
advantage of this model is that it is able to recognize blurred
images and sideways images with some traditional models
which is not seen in such a scenario. The only thing is that
he/she can not ssee through the glasses. In the future, this can
be expanded to see people using video conferencing that will
be useful in obtaining ownership of CCTV cameras that can
enable the police to identify the person in no time. It can also
be used in home security programs.

Fig3.5 Hierarchical Structure

IJISRT21FEB446 www.ijisrt.com 516

Volume 6, Issue 2, February – 2021 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
REFERENCES [15]. Naik, Manoj Kumar, and Aneesh Wunnava. "Classical
2D Face Recognition: A Survey on Methods, Face
[1]. Parkhi, Omkar M., Andrea Vedaldi, and Andrew Databases, and Performance Evaluation." Advances in
Zisserman. "Deep face recognition." (2015). Intelligent Computing and Communication. Springer,
[2]. Abudarham, Naphtali, Lior Shkiller, and Galit Yovel. Singapore, 2020. 375-383.
"Critical features for face recognition." Cognition 182 [16]. Ekmen, Beste, and Hazım Kemal Ekenel. "From 2D to
(2019): 73-83. 3D real-time expression transfer for facial
[3]. Masi, Iacopo, et al. "Face-specific data augmentation for animation." Multimedia Tools and Applications 78.9
unconstrained face recognition." International Journal of (2019): 12519-12535.
Computer Vision 127.6-7 (2019): 642-667. [17]. Singh, Yash, et al. "Method for 3D modelling based on
[4]. Huang, Chen, et al. "Deep imbalanced learning for face structure from motion processing of sparse 2D images."
recognition and attribute prediction." IEEE transactions U.S. Patent No. 10,198,858. 5 Feb. 2019.
on pattern analysis and machine intelligence (2019). [18]. Krishnapriya, K. S., et al. "Characterizing the variability
[5]. Deng, Jiankang, et al. "Lightweight face recognition in face recognition accuracy relative to race." arXiv
challenge." Proceedings of the IEEE International preprint arXiv:1904.07325 (2019).
Conference on Computer Vision Workshops. 2019. [19]. Srivastava, Yash, Vaishnav Murali, and Shiv Ram
[6]. Hau, Cheuk Yu, and Yvonne Sadovy de Mitcheson. "A Dubey. "Hard-mining loss based convolutional neural
facial recognition tool and legislative changes for network for face recognition." arXiv preprint
improved enforcement of the CITES Appendix II listing arXiv:1908.09747 (2019).
of the humphead wrasse, Cheilinus undulatus." Aquatic [20]. Sawant, Manisha M., and Kishor M. Bhurchandi. "Age
Conservation: Marine and Freshwater Ecosystems 29.12 invariant face recognition: a survey on facial aging
(2019): 2071-2091. databases, techniques and effect of aging." Artificial
[7]. Bowyer, Kevin W. "Face recognition technology: Intelligence Review 52.2 (2019): 981-1008.
security versus privacy." IEEE Technology and society
magazine 23.1 (2004): 9-19.
[8]. Mitra, Debasree, Parantapa Sarkar, and Payel Roy. "Face
Recognition by City-Block Distance Classifier in
Supervised Machine Learning." IJRAR DOI: http://doi.
one/10.1729/Journal 21653 (2019).
[9]. Siddiqui, Muhammad Farhan, et al. "Face Detection and
Recognition System for Enhancing Security Measures
Using Artificial Intelligence System." INDIAN
JOURNAL OF SCIENCE AND TECHNOLOGY 13.09
(2020): 1057-1064.
[10]. Connolly, Hannah L., Andrew W. Young, and Gary J.
Lewis. "Recognition of facial expression and identity in
part reflects a common ability, independent of general
intelligence and visual short-term memory." Cognition
and Emotion 33.6 (2019): 1119-1128.
[11]. Dandıl, Emre, and Rıdvan Özdemir. "Real-time Facial
Emotion Classification Using Deep Learning." Data
Science and Applications 2.1 (2019): 13-17.
[12]. Riaz, Sidra, et al. "Age-invariant face recognition using
gender specific 3D aging modeling." Multimedia Tools
and Applications 78.17 (2019): 25163-25183.
[13]. Sawant, Manisha M., and Kishor M. Bhurchandi. "Age
invariant face recognition: a survey on facial aging
databases, techniques and effect of aging." Artificial
Intelligence Review 52.2 (2019): 981-1008.
[14]. Mortezaie, Zahra, and Hamid Hassanpour. "ASurvey ON
AGE-INVARIANT FACE RECOGNITION
METHODS." Jordanian Journal of Computers and
Information Technology (JJCIT) 5.02 (2019).

IJISRT21FEB446 www.ijisrt.com 517