FACE RECOGNITION-BASED ATTENDANCE SYSTEM

Literature Survey
• Priyanka Wagh,Jagruti Chaudhari,Roshani Thakare,Shweta Patil ," Attendance
System based on Face Recognition using Eigen face and PCA Algorithms ",2015.

• Divyansh Methi, Abhishek Chauhan , Divyanshu Gupta," Attendance System

Using Face Recognition",International Journal of Advanced Research in Science,
Engineering and Technology,Vol. 4, May 2017.

• Nandhini R, Duraimurugan N, S P Chokkalingam, “Face recognition based

attendance system”, International Journal of Engineering and Advanced
Technology(IJEAT),ISSN:2249-8958,Volume-8,Issue-3S, February 2019

• Shubhobrata Bhattacharya, Gowtham Sandeep Nainala, Prosenjit Das, Aurobinda

Routray, “Smart Attendance Monitoring System(SAMS): A Face Recognition
Based Attendance System for Classroom Environment”,2018 IEEE 18th
International Conference on Advanced Learning Technologies, Volume-5, 2018
 Objectives
• Detection of unique face image amidst the other natural component such as
walls and other backgrounds.

• Detection of faces amongst other face characters such as beard, spectacles

etc.

• Extraction of unique characteristic features of a face useful in face

recognition.

• Effective recognition of unique faces in a class (individual recognition).

• Automated update in the attendance sheet without human intervention.

• To keep the student updated with their attendance ratio

OpenCV Face Recognition
Introduction
• To build our face recognition system, we’ll first perform face
detection, extract face embeddings from each face using deep
learning, train a face recognition model on the embeddings, and then
finally recognize faces in both images and video streams with
OpenCV.
External Libraries Used are ::
• dlib (obviously)
• face_recognition (which is an easy to use set of face recognition
utilities that wraps around dlib)
 • dlib : We use dlib and OpenCV to detect facial landmarks in an image.
• Facial landmarks are used to localize and represent salient regions of the face,
such as:
• Eyes
• Eyebrows
• Nose
• Mouth
• Jawline
Facial landmarks have been successfully applied to face alignment, head pose
estimation, face swapping, blink detection and much more.

This method starts by using:

• A training set of labeled facial landmarks on an image. These images are manually
labeled, specifying specific (x, y)-coordinates of regions surrounding each facial
structure.
• Priors, of more specifically, the probability on distance between pairs of input pixels.
• Given this training data, an ensemble of regression trees are trained to estimate the
facial landmark positions directly from the pixel intensities themselves (i.e., no
“feature extraction” is taking place).
• face_recognition : (which is an easy to use set of face recognition
utilities that wraps around dlib)

An overview of the OpenCV face recognition pipeline. The key step is a CNN feature extractor that generates
128-d facial embeddings
Process behind dlib’s facial landmark
detector
• The pre-trained facial landmark detector inside the dlib library is used to estimate the location
of 68 (x, y)-coordinates that map to facial structures on the face.
• The indexes of the 68 coordinates can be visualized on the image below:
Process
• Detect faces
• Compute 128-d face embeddings to quantify a face
• Train a Support Vector Machine (SVM) on top of the embeddings
• Recognize faces in images and video streams
 Training Datasets
• To train a face recognition model with deep learning, each input batch of data
includes three images:
1. The anchor
2. The positive image
3. The negative image

• The anchor is our current face and has identity A.

• The second image is our positive image — this image also contains a face of
person A.
• The negative image, on the other hand, does not have the same identity, and
could belong to person B, C, or even Y!
• The point is that the anchor and positive image both belong to the same
person/face while the negative image does not contain the same face.
Optionally we can compute facial landmarks, enabling us to preprocess and align the face.
Face alignment, as the name suggests, is the process of
(1) identifying the geometric structure of the faces and
(2) attempting to obtain a canonical alignment of the face based on translation, rotation, and scale.

While optional, face alignment has been demonstrated to increase face recognition accuracy in some pipelines.

After we’ve (optionally) applied face alignment and cropping, we pass the input face through our deep neural
network:
• The neural network computes the 128-d embeddings for each face and
then tweaks the weights of the network (via the triplet loss function) such
that:
• The 128-d embeddings of the anchor and positive image lie closer together
• While at the same time, pushing the embeddings for the negative image
father away
• In this manner, the network is able to learn to quantify faces and return
highly robust and discriminating embeddings suitable for face recognition.

Here we have 3 classes with 6

images each for describing face
dataset for face recognition with
OpenCV.
 Project Structure
• $ tree --dirsfirst
• .
• ├── dataset (Contains our face images organized into subfolders by name)
• │ ├── adrian [6 images]
• │ ├── trisha [6 images]
• │ └── unknown [6 images]

• ├── images(Contains three test images that we’ll use to verify the operation of our model)
• │ ├── adrian.jpg
• │ ├── patrick_bateman.jpg
• │ └── trisha_adrian.jpg
• ├── face_detection_model (Contains a pre-trained Caffe deep learning model provided by OpenCV to detect faces. This model detects and localizes faces in an
image.)
• │ ├── deploy.prototxt
• │ └── res10_300x300_ssd_iter_140000.caffemodel
• ├── output (Contains my output pickle files. If you’re working with your own dataset, you can store your output files here as well)
• │ ├── embeddings.pickle (A serialized facial embeddings file. Embeddings have been computed for every face in the dataset and are stored in this file)
• │ ├── le.pickle (Our label encoder. Contains the name labels for the people that our model can recognize.)
• │ └── recognizer.pickle (Our Linear Support Vector Machine (SVM) model. This is a machine learning model rather than a deep learning model and it is
responsible for actually recognizing faces.)
•
Summary
Detecting facial landmarks in an image is a two step process:

1. First we must localize a face(s) in an image. This can be accomplished using a number
of different techniques, but normally involve either Haar cascades or HOG + Linear
SVM detectors (but any approach that produces a bounding box around the face will
suffice).

1. Apply the shape predictor, specifically a facial landmark detector, to obtain the (x, y)-
coordinates of the face regions in the face ROI.
Given these facial landmarks we can apply a number of computer vision techniques,
including:
• Face part extraction (i.e., nose, eyes, mouth, jawline, etc.)
• Facial alignment
• Head pose estimation
• Face swapping
• Blink detection
• …and much more!
 Proposed Methodology
● Architecture
 Working

● To bring this system into work, we will need some hardware devices for our
project.
● Firstly, we will need a high definition camera which has to be fixed in the
classroom at a suitable location from where the whole class can be covered in
the camera.
● When the camera takes the picture of all students, that picture is enhanced
for further processing.
● In the enhancement, the picture is first transformed in grayscale image, and
then it will be equalized using histogram technique.
● After enhancement, the picture will be given for detecting the faces of
students which is done by face detection algorithm.
● Then after detection of faces, each student's face will be cropped from that
image, and all those cropped faces will be compared with the database of
faces.
● In that database, all students' information will be already maintained with their
image.
● By comparing the faces one by one, the attendance of students will be
marked on server.
 METHODOLOGY
● For implementing the automated face recognition system, we need to follow some
particular methodologies. The certain steps which need to be followed for successful
attendance marking are as follows:
1) Enrollment
2) Image Acquisition.
3) Converting the image into gray scale Image.
4) Histogram Normalization.
5) Removing Noise.
6) Classification of Skin.
7) Face Detection.
8) Face Recognition.
9) Attendance marking
1) (1)Enrollment:
● The student or person will be enrolled to the database using their general information
and unique biometric features.
● This information will be saved in the form of templates. The enrollment includes:
1. 1.Taking image by camera
2. 2.Enhancement of that image
3. 3.Feature extraction
4. 4.Maintain Database
● The image of person will be captured from the camera and then it will be
enhanced using histogram equalization and noise filtering.
● Then after this process, the features are extracted from the image.
● The unique features will be stored in the face database and a particular id will
be assigned to that person.
1) (2) Image Acquisition:
● A high definition camera device will be installed in front of the classroom.
● The camera device will capture the image of whole classroom.
● This captured image is given as an input to the system.

● (3) Grayscale conversion of image:

● The image which is captured from the camera device sometimes may have the brightness
in it which needs to be removed for the appropriate result.
● Therefore the Captured image is converted to grayscale image for the enhancement
● (4) Histogram Normalization:
● Histogram Normalization is a technique used for contrast enhancement.
● After this the image will be equalized for removing the contrast so that the students sitting
on the back rows can be clearly seen and it will be easy torecognize them.
● Then it generates the histogram of the equalized image.

● (5) Noise Removal:

● When the input image is captured by camera, it may contain the noise which has to be
filtered from image.
● The median filtering is one of efficient techniques for removing the noise.
● (6)Skin Classification:
● In the skin classification technique, all the pixels are made black except the pixels which
are closely related to the skin.
● Those pixels become white.
● The accuracy of face detection algorithm is improved after skin classification.

● (7)Face Detection:
● After the enhancement of image, the image comes to face detection module which will
detect the faces of students from image.
● The Viola and Jones Algorithm is used for the purpose of face detection, also known as
the Ada-Boost algorithm for face detection which is created by Viola P. and M. 1. Jones.
● (8)Face Recognition:
● Face recognition is the next step after face detection.
● The face recognition can be achieved by cropping the faces from the image and comparing
them with the enrolled images in the face database.
● For the face recognition, the concept of selection of region of interest is used, and the faces
are verified one by one using the EigenFace method.

● (9)Attendance:
● After the verification of faces and successful recognition is done, the attendance will be
marked on the server.
Activity diagram of Face Recognition
 ALGORITHM

● ALGORITHM: FACE RECOGNITON FOR ATTENDANCE SYSTEM.

● INPUT: Classroom image captured by the camera.

● OUTPUT: Attendance marking.
● PROBLEM DESCRIPTION: Identification of student
● Step 1: Start
● Step 2: Enroll the students' information in the face database
● Step 3: Install a camera device in classroom.
● Step 4: Input the image taken by camera.
● Step 5: Enhancement of image.
● (1) Convert to grayscale image.
● (2) Generate histogram of grayscale image.
● (3) Equalize the image.
● (4) Generate histogram of equalized image.
● (5) Remove noise from image
● ( 6) Skin classification of image.
● Step 6: Face Detection.
● (1)Crop the faces of students form image
● (2)Select the region of interest.

● Step 7: Face Recognition

● (1)Compare the cropped images with face database images
● (2)Mark the attendance on attendance server.
● (a) If any other face , then
● Go to (2) of step 6
● Step 8: Stop
 Face Detection
● In the face detection step, we will use Viola-Jones algorithm for detecting faces of
students by using haar feature and cascade concept of the Ada-Boost algorithm
The Viola-Jones algorithm is a widely used mechanism for object detection.
The main property of this algorithm is that training is slow, but detection is fast.
This algorithm uses Haar basis feature filters, so it does not use multiplications

Detection happens inside a detection window.

A minimum and maximum window size is chosen, and for each size a sliding step size
is chosen.
Then the detection window is moved across the image as follows:
 Viola – jones algorithm
● (1) Set the minimum window size, and sliding step corresponding to that size.

● (2) For the chosen window size, slide the window vertically and horizontally with the same
step. At each step, a set of N face recognition filters is applied.
● If one filter gives a positive answer, the face is detected in the current window.

● (3) If the size of the window is the maximum size ,stop the procedure.
● Otherwise increase the size of the window and corresponding sliding step to the
● next chosen size and go to the step 2.
 Features

● Each face recognition filter (from the set of N filters) contains a set of
cascade-connected classifiers.
● Each classifier looks at a rectangular subset of the detection window and
determines if it looks like a face. If it does, the next classifier is applied.
● If all classifiers give a positive answer, the filter gives a positive answer and
the face is recognized. Otherwise the next filter in the set of N filters is run.
● Each classifier is composed of Haar feature extractors (weak classifiers).
● Each Haar feature is the weighted sum of 2-D integrals of small rectangular areas
attached to each other. The weights may take values ±1. Below figure shows
examples of Haar features relative to the enclosing detection window.
● Gray areas have a positive weight and white areas have a negative weight.
●
The classifier decision is defined as:

Where,
F is the weighted sum of the 2-D integrals.
m is the classifier number.
t is the decision threshold for the i-th feature extractor.
αm,i and βm,i are constant values associated with the i-th feature extractor.
θm is the decision threshold for the m-th classifier.
 Face Recognition
● .
● For face recognition, we are going to utilize Eigen face technique along with Principle
Component Analysis.
● In the Eigen face, when faces are detected, they are cropped from image. Each
student's face is cropped and the various features are extracted from them like
distance between eyes, nose, outline of face, etc.
● Using these faces as eigen features, the student are recognized and by comparing
them with the face database and their attendance is marked.
 Eigen face

● Eigenfaces are a set of features that characterize the variation between face images
● Each training face image can be represented in terms of a linear combination of the
eigenfaces, so can the new input image
● Compare the feature weights of the new input image with those of the known
individuals
 FACE RECOGNITION PROCESS
One of the simplest and most effective PCA approaches used in face recognition systems
is the so-called eigenface approach.

This approach transforms faces into a small set of essential characteristics called eigen
faces, which are the main components of the initial set of learning images (training set).

Recognition is done by projecting a new image in the eigenface subspace, after which the
person is classified by comparing its position in eigenface space with the position of known
individuals .

The advantage of this approach over other face recognition systems is in its simplicity,
speed and insensitivity to small or gradual changes on the face.
● The whole recognition process involves two steps:
● A. Initialization process
● B. Recognition process
● The Initialization process involves the following operations:
● i. Acquire the initial set of face images called as training set.
● ii. Calculate the Eigenfaces from the training set, keeping only the highest eigenvalues.
● These M images define the face space. As new faces are experienced,
● the eigenfaces can be updated or recalculated.
● iii. Calculate distribution in this M-dimensional space for each known person by projecting
his or her face images onto this face-space.
● Having initialized the system, the next process involves the steps:

● 1) Calculate a set of weights based on the input image and the M eigenfaces by projecting
● the input image onto each of the Eigenfaces.
2) Determine if the image is a face at all (known or unknown) by checking to see if the
image is sufficiently close to a “free space”.
3) If it is a face, then classify the weight pattern as either a known person or as unknown.
● 4) Update the eigenfaces or weights as either a known or unknown, if the same unknown
● person face is seen several times then calculate the characteristic weight pattern and
● incorporate into known faces.
 EIGENFACE ALGORITHM
● Let a face image Γ(x, y) be a two dimensional M by N array of intensity values. Here , consider a set of
image by 200× 149 pixels.
● An image may also be considered as a vector of dimension M × N, so that a typical image of size 200 ×
149 becomes a vector of dimension 29,800 or equivalently a point in a 29,800 dimensional space.
● Step1: Prepare the training faces
● Obtain face images I1 , I2 , I3, I4 , . . . . . . ,IM (training faces).
● The face images must becentered and of the same size.
Step 2: Prepare the data set
● Each face image Ii in the database is transformed into a vector and placed into a training set S.
● S=Γ1,Γ2,Γ3,Γ4.......ΓM
● Each image is transformed into a vector of size MN × 1 and placed into the set.
● For simplicity, the face images are assumed to be of size N × N resulting in a point in dimensional space.
● Step 3: ompute the average face vector
● The average face vector (Ψ) has to be calculated by using the following formula:
●

● Step 4: Subtract the average face vector

● The average face vector Ψ is subtracted from the original faces Γ and the result stored in the Φ variable
●

●
 ● Step 5: Calculate the covariance matrix
● We obtain the covariance matrix C in the following manner,

∙ Step 6: Calculate the eigenvectors and eigenvalues of the covariance matrix

∙ The covariance matrix C in step 5 has a dimensionality N²xN² , so one would have N²
∙ eigenface and eigenvalues.
● Compute the eigenvectors uᵢ of .The above matrix is very large which is not practical!!!

Step 6.1: Consider the matrix

Step 6.2: Compute eigenvectors and finally we get

● Step 7: keep only K eigenvectors (corresponding to the K largest eigenvalues)
● Eigenfaces with low eigenvalues can be omitted, as they explain only a small part of characteristic
● features of the faces.

● Next we have to project the training sample into the Eigenface space. The feature weight for the training images
can be calculated by the following formula:
Schematic diagram
Flow chart
Thank You