Implementation of Smart Attendance on

FPGA
Athira Shanker, Debolina Roy, Mebin P M, Sivasankaran K.

M.Tech VLSI Design, Dept. of Micro & Nano electronics, SENSE, Vellore Institute of Technology, Vellore, Tamil Nadu,
India

E-mail: {athira.shanker2019, debolina.roy2019, mebinpm.2019}@vitstudent.ac.in, [email protected]

Google images which is a new invention of Google allows

Abstract— Manual attendance has become a very tiring job as the us to compare with an image stored in the web data set to the
number of people in every workplace or class has increased. Along with image given by the user. If a person directly provides the
these difficulties faced, it has proved to be less efficient and time image in the search box or applies the URL of the image it
consuming. This poses a lot of issues in situations where there are will detect the similarity of images and it will verify and
attendance criteria. In our paper we are aiming to tackle this problem compare the images present in the data set. Photographers can
with the help of a system for smart attendance. The system detects faces check by this whether their images have been used without
real time and matches it with an existing database to recognize each their knowledge.
face. This type of smart attendance system reduces the need for
spending time for attendance. We use Altera Cyclone-IV FPGA to
Face recognition algorithm, iris recognition algorithm,
implement this system.
speech recognition algorithm gives an authenticate
multibiometric system that is needed these days. Many face
Keywords— smart attendance, face detection, viola jones recognition and detection algorithms have been discovered
algorithm, face recognition, FPGA like LBP generation and AdaBoost algorithm, Linear
discriminant analysis, discrete wave transform, viola jones and
I. INTRODUCTION principal component analysis (PCA), RCT based segmentation
is there of which Viola Jones for face detection and PCA for
face recognition plays a key role.
In today’s arena where the human beings are in need of
more protective work environment so new procedures are in The reason for choosing viola Jones algorithm for face
lot of demand. For security purposes legitimate users should detection is the high detection rate that this algorithm
be given proper access and unauthorized users should be provides.PCA can be considered the best for its efficiency in
denied. dimensionality for face recognition.[5] Another reason for
Smart attendance systems like Face detection and biometric choosing Viola Jones method is that it detects a full view of
finger print nowadays are ideal tools that provide a smart frontal face by providing precise instruction to the
security system [9]. computer.[1]
The process by which we locate the position of faces in an The proposed work for real-time face detection based on
image is known as face detection. The changes in skin colour, Viola Jones has been designed using Verilog and we intended
background features and facial expressions make this task to implement it on Quartus and DE2-115 FPGA Board.
difficult and cumbersome. Security system, image search and
photography are the three major areas where we can apply The paper is prepared in six sections that will include
face detection.[8] introduction, methodology, face recognition, face detection,
implementation and results.
Auto focus is used these days in every digital camera for
face detection which are programmed to capture automatically
whenever the smile of user is detected.
II. METHODOLOGY
Block diagram of our proposed work is given below:
 III. FACE DETECTION ALGORITHM
In our proposed work we have used Viola Jones algorithm
to know the location of an image (face detection). This
algorithm will extract the possible of different faces using an
integral image window and the average of these extracted
features will be stored in a memory location. This algorithm
will eliminate all those images which are not stored in
database by using a cascade structure. Every stage will consist
of number of Haar features and are classified by the classifier.
A stage comparator is used to give the outputs of the classifier.
This stage comparator will sum up all the outputs and compare
this summed up value with a threshold value which will
decide whether the stage can be passed or not. [2]
Fig.1 Proposed Block Diagram

A. Terasic TRDB-D5M Camera Module

Fig.2 Terasic TRDB-D5M CMOS camera module

The Terasic TRDB-D5M camera is CMOS technology-

based module that works on 3.3V supply. It has micron 5 MP
CMOS sensor. It has a resolution of 2592x1944 pixels with a
maximum frame rate 15fps. The output of the module is in
RGB Bayer Pattern format. Fig.2 shows the TRDB-D5M
module.

B. The Altera DE2 - 115 Development Board

Fig.4 Viola Jones Algorithm

A. Haar Feature Selection
Every human face has some related highlights where the
upper cheeks are less bright than the eye area or the nose area
is brighter than the eye area. This data is utilized to develop
certain highlights or features known as the Haar features.
Many stages are constructed using these similar features and
the stages are not limited to number of features.

Each face image is captured and converted from RGB to

grayscale format. This grayscale image is scanned and Haar
Fig.3 Altera DE2 - 115 Development Board features of the current stage is searched which includes two to
three rectangles. The Haar feature values are calculated by
Fig. 3 shows the DE2-115 development board. The board calculating the area of each rectangle and multiplying weights
has Altera Cyclone IV FPGA which can be programmed to with this area. The weight calculation is done by the AdaBoost
implement various hardware blocks. It has on chip SRAM, algorithm as shown in Fig.5. [[1]-[2]]
flash memory, VGA port, seven segment displays and GPIO
pins. GPIO pins are used to connect the camera module on the
board.
 window to detect error and the process will be repeated until it
completes all the mentioned stages. [1]- [2](Fig.6)

IV. FACE RECOGNITION ALGORITHM

Fig.5 Haar Feature Classifiers For our work we have used Principal component analysis
(PCA) algorithm for face recognition. Principal component
analysis is one of the efficient and successful techniques for
B. Integral Image
image compression and recognition. It is a measurable strategy
Computing the rectangular features at constant speed is under the expansive title of factor analysis. PCA lessens the
done by the integral image which overcomes the problem of broader dimensionality of information space to smaller
inefficiency when large set of images is given as input. intrinsic dimension. The effective and simplest approach used
Integral image value at any area (x, y) is characterized as the by PCA is the Eigen face approach. PCA transforms the face
aggregate of pixels which are available above and to the left of images into Eigen subspaces. Whenever a new face image is
that area incorporating with the pixel present at xth and yth received the Eigen vector of the face image is compared with a
area.[2] mean Eigen vector (measuring the Euclidean distance). A
large input of facial vectored image can be handled only by
small Eigen weights. Each of these face vectors gets stored in
a memory face space. And whenever an input image is
received it gets projected in the face space.

The benefit of this methodology over other face

acknowledgment frameworks is in its straightforwardness,
speed and lack of care toward little or slow changes on the
face.

The methodology in PCA is as follows:

Fig.6 Integral Image Formation 1. Creating a training set and loading it with M face
images. The face images must be of same size and
centered.
C. AdaBoost training 2. Converting the face images in the training set to face
vectors.
Single Haar features having weak discrimination capability 3. Normalizing the face vectors i.e. removing the
cannot satisfy all the essentials needed for an object detection. common features that these faces share together.
If we consider every Haar features having variation in position 4. Decomposition of covariance matrix for calculation
and size then calculated features will become thousands and of Eigen vectors.
hundreds. Among these thousands of features some will 5. Dimension reduction to reduce calculation and effect
provide higher range of values on the face crest. of noise.
6. Retrieving the original dimensionality of the face
To resolve such issue Viola jones uses an ensemble from the lower dimension vector of the reduced
learning algorithm known as Adaboost that takes (combines) a dimension matrix.[10]
collection of base learners or weak classifiers and produces a 7. Each face of the training set can be represented as a
strong classifier. The strong classifiers will produce better weighted sum of K Eigen vectors + mean face image.
results. Every weak classifier act as a threshold for the single
Haar like feature which decides the sub-window. A face image
consists of different shades and scale values. Each classifier
will consist of is 2 rectangular areas with in which the dark
and light spots of the faces is calculated by combining all the
grayscale values. The combined value is calculated and
compared with the thresholds. If the accumulated value
exceeds the strong threshold the current window will be closed
and a new sub window will open for re-evaluation. This will
continue till the last stage. [1]

D. Cascade Classifier
Fig.7 PCA Block Diagram
The main function of the classifier is to store the Haar
features in number of stages to increase the speed of detection.
A face is detected only if a sub window passes all the
mentioned stages. A classifier will detect the errors of the
previous stage and will assign weights to those features which
are more erroneous. It will then pass on to the next sub
 V. IMPLEMENTATION

A. MATLAB Implementation of Face Detection

To understand and study the basic functionality of the

algorithm, MATLAB was used. The Viola Jones algorithm for
face detection was implemented with the help of various in-
built functions in MATLAB. The image was captured using
the laptop’s webcamera and then processed to get the output.
The output showed detected faces with a box drawn such that
it denotes the detected faces.

B. Implementation of Video Capture and Display

The implementation of video capture and display was

implemented using Quartus. This was done with help of
Platform Designer tool in Quartus. In Platform Designer,
various IP modules were added to perform the function of
video capture and display. Fig.9 Hardware setup for video capture and display
C. Verilog Implementation of Grayscale conversion

It is a known fact that a Verilog module cannot take an

image directly as the input. Due to this, we initially converted
the bitmap (.bmp) image into hexadecimal (.hex) format with
the help of MATLAB. This hexadecimal file is then given as
the input the Verilog module for RGB to grayscale format.

The input RGB was a stored into a 24-bit mxn memory

model using $readmemh command. Further, manipulations
were done to segregate Red, Blue and Green components. This
was then converted into 8-bit grayscale output.

The grayscale image is then written into an output

Bitmap image.

VI. RESULTS
Viola Jones algorithm was implemented in Fig.10 Displaying captured video on a VGA monitor
MATLAB (Fig.8).
Verilog code for reading an image from a specified
memory location in RGB format was implemented. The
read RGB image was then converted into grayscale format
and stored into another Bitmap image.

Fig.8 Implementation of Viola Jones Algorithm in

MATLAB

Video capture and display on VGA monitor was

implemented on Altera DE2-115 board (Fig.9).

Fig.11 RGB image read from specified memory location

 Fig.12 Output image after grayscale conversion

REFERENCES
[1] Alahmadi and S. M. Qaisar, “Robust Real-time Embedded Face
Detection Using Field Programmable Gate Arrays (FPGA),” 2019
Advances in Science and Engineering Technology International
Conferences (ASET), 2019.
[2] S. V. Chakrasali and S. Kuthale, “Optimized face detection on
FPGA,” 2016 International Conference on Circuits, Controls,
Communications and Computing (I4C), 2016.
[3] Raghuwanshi and P. D. Swami, “An automated classroom
attendance system using video based face recognition,” 2017 2nd
IEEE International Conference on Recent Trends in Electronics,
Information & Communication Technology (RTEICT), 2017.
[4] S. Poornima, N. Sripriya, B. Vijayalakshmi, and P. Vishnupriya,
“Attendance monitoring system using facial recognition with audio
output and gender classification,” 2017 International Conference
on Computer, Communication and Signal Processing (ICCCSP),
2017.
[5] L. Schaffer, Z. Kincses, and S. Pletl, “FPGA-based low-cost real-
time face recognition,” 2017 IEEE 15th International Symposium
on Intelligent Systems and Informatics (SISY), 2017.
[6] V. P. Korakoppa, Mohana, and H. V. R. Aradhya, “An area
efficient FPGA implementation of moving object detection and
face detection using adaptive threshold method,” 2017 2nd IEEE
International Conference on Recent Trends in Electronics,
Information & Communication Technology (RTEICT), 2017.
[7] Ahmad, A. Amira, P. Nicholl, and B. Krill, “FPGA-based IP cores
implementation for face recognition using dynamic partial
reconfiguration,” Journal of Real-Time Image Processing, vol. 8,
no. 3, pp. 327–340, 2011.
[8] D. N. Arya, S. K.l.v., R. Reddy, S. S, and S. K, “A face detection
system implemented on FPGA based on RCT colour
segmentation,” 2016 Online International Conference on Green
Engineering and Technologies (IC-GET), 2016.
[9] Liton Chandra Paul, Abdulla Al Sumam, “Face Recognition Using
Principal Component Analysis Method,”International Journal of
Advanced Research in Computer Engineering & Technology
(IJARCET) Volume 1, Issue 9, November.
[10] N. Çevik and T. Çevik, “A novel high-performance holistic
descriptor for face retrieval,” Pattern Analysis and Applications,
2019