Human Recognition

using Face

Yashvi 2021UCD2117
Ritul 2021UCD2127
Meenal 2021UCD2128
Bhumika 2021UCD2167
Step 1: Face detection

The camera detects and locates the image of a face, either alone or in a crowd.
The image may show the person looking straight ahead or in profile.

Firstly, humans can be very different — skin color, haircuts, and tattoos change
their looks and affect the accuracy of the detection. Secondly, angles matter — a
face can be very different depending on the viewpoint of the camera. Thirdly,
different facial expressions can change the way a person looks. Finally, there are
matters of lighting and occlusion, as the face might be obstructed by something
or the picture can be taken in relative darkness.

Some of the Face Detection algorithms are :Yolo, SSD, HOG and HAAR
cascade, SIFT
Step 2: Face analysis

Next, an image of the face is captured and analyzed. Most facial

recognition technology relies on 2D rather than 3D images because it
can more conveniently match a 2D image with public photos or those in
a database.

The software reads the geometry of your face. Key factors include the
distance between your eyes, the depth of your eye sockets, the distance
from forehead to chin, the shape of your cheekbones, and the contour of
the lips, ears, and chin. The aim is to identify the facial landmarks that
are key to distinguishing your face.
Step 3: Converting the image to data
The face capture process transforms analog information (a face) into a
set of digital information (data) based on the person's facial features.
Your face's analysis is essentially turned into a mathematical formula.
The numerical code is called a faceprint. This faceprint serves as a
unique identifier for the individual's face, similar to a fingerprint for
biometric identification. The faceprint captures the essential
characteristics of the face in a concise and standardized format.
Step 4: Finding a match
Your faceprint is then compared against a database of other
known faces. For example, the FBI has access to up to 650
million photos, drawn from various state databases. On
Facebook, any photo tagged with a person’s name becomes
a part of Facebook's database, which may also be used for
facial recognition. If your faceprint matches an image in a
facial recognition database, then a determination is made.

Of all the biometric measurements, facial recognition is

considered the most natural. Intuitively, this makes sense,
since we typically recognize ourselves and others by looking
at faces, rather than thumbprints and irises. It is estimated
that over half of the world's population is touched by facial
recognition technology regularly.
 APPLICATIONS
● Unlocking phones:use face recognition to unlock the device.
● Law enforcement
● Airports and border control : Facial recognition has become a familiar sight at many airports
around the world. Increasing numbers of travellers hold biometric passports, which allow them to
skip the ordinarily long lines and instead walk through an automated ePassport control to reach the
gate faster.
● Finding missing persons : Facial recognition can be used to find missing persons and victims of
human trafficking. Suppose missing individuals are added to a database. In that case, law
enforcement can be alerted as soon as they are recognized by face recognition — whether it is in
an airport, retail store, or other public space.
● Reducing retail crime : Facial recognition is used to identify when known shoplifters, organized
retail criminals, or people with a history of fraud enter stores. Photographs of individuals can be
matched against large databases of criminals so that loss prevention and retail security
professionals can be notified when shoppers who potentially represent a threat enter the store.
Face Detection Algorithms in Still Photos
and Videos
SIFT
CNN
Residual blocks
Bounding box regression
IOU, Non-Max Suppression. YOLO
 Facial Recognition
Facial recognition is a way of identifying or confirming an
individual’s identity using their face.Among the various
biometric ID methods, the physiological methods
(fingerprint, face, DNA) are more stable than methods in
behavioral category (keystroke, voice print).

Many people are familiar with face recognition technology

through the FaceID used to unlock iPhones (however, this is
only one application of face recognition). Typically, facial
recognition does not rely on a massive database of photos to
determine an individual’s identity — it simply identifies and
recognizes one person as the sole owner of the device, while
limiting access to others.Beyond unlocking phones, facial
recognition works by matching the faces of people walking
past special cameras, to images of people on a watch list.
Local binary pattern (LBP)
It’s a technique used to describe the texture or patterns in an image. It is a texture
descriptor that computes a binary code for each pixel in an image based on the
intensity values of its neighboring pixels. Following is the procedure used in LBP:
a. The face area is first divided into small regions/blocks.
b. Choose a pixel in the image and a neighborhood (e.g., 3x3, 5x5, etc.) around it.
c. It characterizes the local texture patterns by comparing the intensity of a central pixel
with its surrounding pixels.
d. Threshold the pixel values in the neighborhood using the intensity of the central pixel.
e. Encode the binary result as a binary number (e.g., clockwise or counterclockwise) to
represent the local texture pattern.
f. After computing LBP for each pixel in the image, a histogram of the LBP codes is
created.
g. This histogram represents the distribution of different local texture patterns in the
image. i.e., by computing a histogram for each of the cells, we actually are able to
encode a level of spatial information such as the eyes, nose, mouth, etc., that we
would otherwise not have.
·
These feature vectors are then compared using similarity measures (e.g., Euclidean distance, cosine
similarity) to perform face recognition.
Advantages of LBP:

1. LBP is robust to changes in illumination, facial expressions, and small variations in pose.
2. It captures local texture details, which are important for distinguishing between different
facial regions.
3. LBP is computationally efficient and can be computed rapidly, making it suitable for real-
time applications.

Disadvantages of LBP:

1. LBP may not capture global facial features effectively, especially in cases of significant
pose variations or occlusions.
2. It may require additional techniques or feature fusion approaches to improve recognition
accuracy in complex scenarios.
Fisher Faces using LDA(linear
discrimnant analysis)
Linear Discriminant Analysis (LDA) is a dimensionality reduction technique commonly used in face
recognition to extract discriminative features from still images. The main goal of LDA in face recognition is to
project the high-dimensional face image data into a lower-dimensional subspace while maximizing the class
separability. Each face image in the dataset is typically represented as a high-dimensional feature vector,
where each element corresponds to a pixel or a feature extracted from the image. LDA method overcomes
the limitation of Principle Component Analysis method by applying the linear discriminant criterion.
Steps in LDA:

a. Each face image in the dataset is typically represented as a high-dimensional feature

vector, where each element corresponds to a pixel or a feature extracted from the image.
b. Calculate the mean vector for each class (e.g., each person's face in the dataset) by
averaging the feature vectors of all images belonging to that class.
c. Compute the within-class scatter matrix (Sw), which represents the spread of data within
each class. It is calculated as the sum of covariance matrices for each class.
d. Compute the between-class scatter matrix (Sb), which measures the separation between
different classes.
e. Compute the generalized eigenvalues and eigenvectors of the matrix.
f. Sort the eigenvalues in descending order and select the top L eigenvectors
corresponding to the largest eigenvalues to form a transformation matrix W.
g. The transformation matrix W projects the high-dimensional feature vectors onto a lower-
dimensional subspace while maximizing class separability.
h. After obtaining the transformation matrix W through LDA, face recognition is performed
by projecting face images onto the subspace defined by W.
Advantages of LDA:

● LDA maximizes the class separability, making it effective for discriminating between
different individuals in face images.
● It reduces the dimensionality of the feature space, which can improve computational
efficiency and reduce the risk of overfitting.
● LDA is particularly useful when the number of training samples is limited compared to
the feature dimensionality.

Disadvantages of LDA:

● LDA assumes that the data follows a Gaussian distribution and that the classes have
equal covariance matrices, which may not always hold true in practice.
● It requires labeled training data to learn the discriminant features, making it a supervised
learning technique.
YOLO FOR FACE RECOGNITION
Face recognition in videos
 Using OpenCV and Deep Learning (OpenFace)

1. Detect faces
2. Compute 128-d face embeddings to quantify a face
3. Train a “standard” machine learning classifier (SVM,
SGD classifier, Random Forest, etc.) on top of the face
embeddings, and therefore get a OpenCV face
recognition pipeline.
4. Recognize faces in images and video streams
To train a face recognition model with deep learning, each input batch of data includes three
images:

1. The anchor
2. The positive image
3. The negative image

The anchor is our current face and has identity A.

The second image is our positive image — this image also contains a face of person A.

The negative image, on the other hand, does not have the same identity, and could belong
to person B, C, or even Y!

The point is that the anchor and positive image both belong to the same person/face while
the negative image does not contain the same face.
Knowledge-based methods

1. use information about the face, its features, shape, texture, or skin color.
2. set of rules is distinguished that a frame fragment must meet to be considered a human face.
Optical Flow Features:

● Motion-based Embeddings: Compute optical flow

features to capture motion information between
consecutive frames. These features can be combined
with appearance-based embeddings to encode both
spatial and temporal information.
 3D Convolutional Networks:
● 3D CNNs: Extend traditional CNNs to operate on
spatiotemporal volumes of video data. 3D CNNs can
directly extract embeddings from video sequences
while capturing both appearance and temporal
information.
facial recognition encoding from facial recognition and store it in pickle file.You can
read this encoding again and match against the faces in image and video file.If it is
match then it will throw known_name which we stored in program.
The neural network computes the 128-d embeddings for each face and then tweaks the
weights of the network (via the triplet loss function) such that:

1. The 128-d embeddings of the anchor and positive image lie closer together
2. While at the same time, pushing the embeddings for the negative image father away

And then train a “standard” machine learning classifier (SVM, SGD classifier, Random Forest,
etc.) on top of the face embeddings.In this manner, the network is able to learn to quantify faces
and return highly robust and discriminating embeddings suitable for face recognition.
 Evaluation of face recognition techniques

Finding the best detector lies on the preference of the users and
what configurations give us the best balance of speed and
accuracy that a particular application will require.
Principal Component Method
recognition process to construction of a certain number of principal components of images for an input
image. However, in cases where there are significant changes in illumination or facial expression in the
face image, the effectiveness of the method is significantly reduced.

Convolutional Networks
CNN provides partial resistance to scale changes, shifts, rotations, changes in angle, and other distortions.
CNN was developed in DeepFace, which was acquired by Facebook to recognize the faces of its social
network users

LBP
Invariant to small changes in lighting conditions and small image rotations.. Among the disadvantages is
the need for high-quality image preprocessing due to high sensitivity to noise, as the number of false binary
codes increases in its presence.

Neural networks Based methods —- include addition of a new reference face to the database,
which requires complete retraining of the network on the entire available set, and this is a rather
lengthy procedure that, depending on the size of the sample, requires hours of work or even several
days.