Multi-Faces Recognition Process

Using Haar Cascades and Eigenface Methods

Teddy Mantoro*, Media A. Ayu Suhendi
Faculty of Engineering and Technology Faculty of Computer Science
Sampoerna University, Jakarta, Indonesia. Mercu Buana University, Jakarta, Indonesia
{teddy,media}@ieee.org [email protected]

Abstract—Face recognition can be considered one of the most detector will test each part of the image and classify it as "face"
successful biometric identification methods among several types or "not face". This facial classification uses a fixed scale, for
of biometric identification including fingerprints, DNA, palm example 50 × 50 pixels. If the face of the image is larger or
print, hand geometry, iris recognition, retina and odour/scent. smaller than the fixed scale, the classifier continuously
Face recognition provides biometric identification that utilizes processes the image, searching for a face. The classifier can use
the uniqueness of faces for security purposes. The problem with data stored in an XML file to decide how to classify each
face recognition using biometric identification is its lengthy image location.
process and the accuracy of the results. This paper proposes
solutions for a faster face recognition process with accurate The next part of this paper presents previous works that are
results. The proposed face recognition process was done using a related to face recognition, especially ones that can identify
hybrid process of Haar Cascades and Eigenface methods, which multiple faces. It is then followed by the methodology used in
can detect multiple faces (55 faces) in a single detection process. the study presented in this paper. Later, the results of the
This improved face recognition approach was able to recognize experiments conducted are presented and discussed in section
multiple faces with 91.67% accuracy level. IV. Then, the conclusion of this study can be found in section
V.
Keywords- Face recognition; biometric identification; fast face
recognition process; Haar Cascades method; Eigenface method.
II. RELATED WORK
I. INTRODUCTION Face recognition is one of the fastest growing research
areas and it has been used in many domains. In the field of
Face recognition has been used as an authentication process
education, it is shown to have the ability to detect, analyze and
in various fields and especially in computer security related
process emotions in order to get positive teaching effects such
activities, such as homeland security, building access security,
as perception, understanding and expressing emotions [1]. This
criminal identification, as well as user identification in small
approach consists of three stages: Feature extraction, subset
mobile devices. Face recognition also plays a significant role in
feature and emotion classifier. A Haar Cascades method is used
the research field of biometric and computer vision. The goal
to detect the input image, a face, as the basis for the feature
of a face recognition system is to have a negligible
extraction of the eyes and mouth, and then the Sobel edge
misclassification rate. Biometric technology is used for
detection is used to obtain the characteristic value [1].
authenticatition and it may analyze human behavior. Each
biometric system has its own advantages and disadvantages, so There are two major approaches for feature extraction,
that proper consideration is required when selecting one to use typically holistic feature and local feature. In the holistic
in an application. feature-based approach, the features are extracted from the face
as a whole, which may sometimes be affected by occlusion and
Face detection algorithms are considered to require intensive
expression changes. Whereas in local features-based
computation, which makes it is difficult to perform face
approaches, these are overcome as patches of the image are
detection. Most of the face recognition processes are
considered. Also, they are scale and rotation invariant [2]. This
implemented on a single face at a time. For a single face to be
study used the holistic feature approach using eigenface.
recognized, it may need a short period of time, but for many
faces/people, using single face recognition, it will take a lot of Facial recognition systems have been used in small mobile
time. Thus, it is necessary to develop a system for multiple environments as well, to recognize images and video. It uses
faces recognition in one go to speed up the recognizing multiple object detection using the Viola-Jones cascade
process. This paper presents a study on multiple faces classifier in the OpenCV library [3]. A frontal face is good for
recognition using a hybrid method of Haar Cascades and face recognition accuracy. Non-frontal facial images can be
Eigenface.This study aims to improve the performance of face reconstructed to frontal face images to increase the accuracy
recognition process using the Haar Cascades and Eigenface of the facial recognition [4]. Another way to increase the
method. accuracy of the face recognition can be through filtering
OpenCV can be used as a face detector type that works techniques [5]. As well the similarity measurement can be
with the Haar-cascade classifier. From an image, a face used in face recognition by utilizing the probabilistic

978-1-5386-6220-5/18/$31.00 ©2018 IEEE

histograms of the maximum likelihood manner [6]. Robust The next stage is the feature extraction process, where
face detection based on Viola Jones face detection algorithm important features for the face recognition are extracted from
[7, 8, 9] can also be implemented using a graphics processing the pre-processed images. The Eigenface method is combined
unit (GPU) [7, 8, 10]. with PCA to perform the feature extraction process for the
The implementation model of this study is to help the training data. The test data can be directly processed using
security control. This approach has also been implemented by PCA for their feature extraction.
detecting face as in [1, 7, 8, 9]. This study also conducted Following the feature extraction process is the recognition
multiple face detection tasks such as in [9,10] using sequenced process, where the PCA_train and PCA_test data will go
images. through the similarity distance calculation based on the
In video-based face recognition systems, the illumination Euclidian distance technique. Based on the Euclidian distance
and pose variation problems are predominant. Most of the calculations, the result for the recognized image (read: face) is
efficient face recognition systems are developed for controlled concluded.
or indoor environments, hence some fail to give accurate In the implementation of the proposed face recognition
recognition in outdoor environments with variations in process, as shown in Figure 2, it begins with capturing images
illumination [2,8]. A video-based face recognition system using a webcam to get training data, which then goes through
based on the Viola Jones face detection algorithm is an the face detection process, feature extraction process, and then
essential building block in many applications, such as video the core of the face recognition to get the results. The results
surveillance and tracking [7, 8, 9]. are sent to the database. The training data in the database is
later used in the face recognition process of the test data from
III. METHODOLOGY the registration on the day of the exam to authenticate the
The face recognition process for multi-faces that is participants of the exam prior to accessing the exam questions
proposed in this study is depicted in Figure 1. This face in the computer system.
recognition process is going to be implemented in the
registration of computer-based exams. The process flow of the
facial recognition during registration is shown in Figure 2.

Figure 1. The proposed face recognition process

The proposed process is started with pre-processing of the
Figure 2. The implementation of the face recognition process on
training data by converting the RGB images to gray-scale
the exam day
images and reducing the images to 8-bits color. Then
implementing the Haar Cascade method to the images by going During the face detection process the webcam will detect
through the Haar feature phase, integral image and cascade the face by putting boxes to the detected faces as can be seen in
classifier. Figure 3.
 Figure 5. Finding the Integral Image

One example of counting of Integral Image is as follows:

Figure 3. Face detection process on the image
SUM: (D + A) - (B + C)
A. Haar Cascade Technique The Haar feature value is calculated based on the Integral
The technique starts with the Haar feature phase where face image value that is defined as the distinguishing value
detection is performed by using a higher differentiator between between the box (white and black pixels), which is usually
‘face’ and ‘not face’. known as the threshold value. This threshold is a parameter
to determine whether the object is a face or not.
As shown in Figure 4, there are rectangular features on
some parts of the image which are in two forms: black (dark) The Cascade Classifier process combines many weak
and white (bright). Based on these rectangles, the Haar-like classifiers to become sharper images (strong classifiers) by
feature is calculated. The Haar-like feature is derived from the assigning weights to the weak classifiers image. Cascade
difference between the sum of pixels of the dark area and the Classifier indirectly this can be said as a filter chain. Each
sum of pixels of the bright area, as expressed in the following filter is a classifier by a separate boost with a relatively small
formula: number of weak classifiers.

Where:
= sum of pixels of the dark area
= sum of pixels of the bright area
= the Haar-like feature
When the Haar-like feature is higher than a certain
threshold, it can be stated that a face or faces are within the
area. To efficiently filter a high number of faces within the
image, an integral image technique is used.

Figure 4. An image with Haar-like feature Figure 6. Process flow of Cascade Classifier
The Haar feature can be calculated based on the Integral The threshold value for each filter is set whenever the
image. Integral image is a technique for calculating the filter process occurs, so the value of the threshold is
feature value quickly by changing the value of each pixel dynamic. Filters at each level have been trained to classify
into a new image representation. The integral image value is imagery that has passed the previous stage. During the
the pixel accumulation value of the top to left calculation. classification process, if one of the filters failed to pass, then
For example, the pixel value (a, b) has an accumulative value the image can be said to be a non-face. If the image is not
of pixels (x, y). After integrating, the pixel at (x,y) contains enough, means a strong classifier, then the process is
the sum of all pixel values on the shaded rectangle. The pixel repeated until the weights are met by raising the value of the
values in rectangle D is ((x4,y4) + (x1,y1)) – ((x3,y3) + threshold.
(x2,y2)).
When the image can pass through each filter in the chain,
it means that the area is the face. To get accurate values, the
processing of haar-like features is performed. If at stage-1 D3 120 120
the results do not meet the criteria, the result is rejected. D4 170 170
Then, the algorithm will move to sub next window that is D5 200 200
phase-2 and will count feature back, if the result received fits
in the threshold, then it will be continued at the next filter
stage so that the sub window step decreases and moves The next experiment is to see how many different faces can
closer to the real image value. be detected in a single instance during the face recognition
process. The results are presented in Table 3.
Filters at each level are trained to classify previously
filtered images (training set is a database of faces). During The data in Table 3 shows that out of 60 faces that need to
its use, if one of the filters fails, that region of the image is be detected, 55 can be successfully recognized. This means
classified as "Not Face". When the filter successfully passes that the success rate is 91.67% for the multi-faces recognition
that region, it then enters the next filter. Image regions that process.
have been through all filters will be considered as "Face".
The face detection program is processed as follows:
1. The face image of the capture result is changed from Table 3. Results of multi-faces recognition process
RGB to Greyscale.
Location Targeted Detected
2. Normalization of lighting. number of number of
faces faces
3. Haar Cascade detection. Classroom 1 2 2
4. Facial extract with eigenface calculation for face Classroom 2 3 3
recognition. Classroom 3 5 5
Classroom 4 6 6
5. Face recognition process with PCA calculation Open space 60 55
(composing flat-vector image matrix and determining the
vector of the image) and the calculation of eigenvector
and eigenvalue (by computing co-variance). Figure 6, shows the comparison of multi-faces recognition
process.
IV. RESULT AND DISCUSSION
Experiments using the proposed face recognition process
involving several factors have been conducted to see the
performance of the system developed based on the proposed
technique.
The first experiment was to look at the position factor of
the faces that were being recognized. Data in Table 1 shows
the results from this experiment. The results show that only
when the face is in the side facing position for 30 degrees, then
the face was fail to be detected.
Table 1. Detection results for various face position
Position Degree of Detection
the position results (%)
Normal 0 100
Head up 25 100 Figure 6. Comparison of multi-faces recognition process
Head down 25 100
Side facing 1 15 100 The data in Table 4 shows the following:
Side facing 2 30 0 1. The facial recognition process is successfully optimized
within the specified target.
The second experiment studied the distance factor of the
face detection process. Table 2 shows the results of this 2. When the face is turned away, the facial image is still
experimentation. The data shows that face detection can be able to be recognized, even though the selected method is for
successfully performed up to 200 cm. straight face detection.
Table 2. Detection results at various distance Table 4. Multi-face recognition optimization

Distance Targeted Detection Face Position Image The results

distance distance targets to be of the face
D1 20 20 recognized image to be
D2 70 70 recognized
1 facial image 1 1
 (normal) The facial recognition process with the Haar Cascade and
2 facial image 2 2 Eigenface method can be successfully performed at a distance
(optimization) of more than 200 cm using a webcam.
3 facial image 3 3
(optimization) For future work, while the current development is for
straight faces, it can to be improved to recognize faces at
varying angles. Followed by the optimization of the facial
Figure 7 shows that during test 1, 2 and 3, the face of user-1 recognition process for use on a small mobile device.
and the face of user-2 can be recognized. Recognizing the
facial image can be performed successfully up to 200 cm. REFERENCES

[1] D. Yang, Abeer Alsadoon, P.W.C. Prasad, A.K. Singh, A. Elchouemi,

“An Emotion Recognition Model Based on Facial Recognition in Virtual
Learning Environment”, Procedia Computer Science, Vol. 125, pp.2-10,
2018.
[2] V. Mohanraj, M. Vimalkumar, M. Mithila, V. Vaidehi, “Robust Face
Recognition System in Video using Hybrid Scale Invariant Feature
Transform”, Procedia Computer Science, Vol. 93, pp. 503-512, 2016.
[3] N. Kazanskiy, V. Protsenko, P. Serafimovich, “Performance analysis of
real-time face detection system based on stream data mining
frameworks”, Procedia Engineering, Vol. 201, pp. 806-816, 2017.
[4] J. Kavitha, T.T. Mirnalinee, “Automatic Frontal Face Reconstruction
Approach for Pose Invariant Face Recognition”, Procedia Computer
Science, Vol. 87, pp. 300-305, 2016.
[5] Vinay A., G. Kathiresan, D. A. Mundroy, H. Nihar Nandan, C. Sureka,
K.N. B. Murthy, S. Natarajan, “Face Recognition using Filtered Eoh-
sift”, Procedia Computer Science, Vol. 79, pp. 543-552, 2016.
Figure 7. Results of facial image recognition based on distance.
[6] G. Srikote, A. Meesomboon, “Face Recognition Performance
Improvement Using Derivative of Accumulated Absolute Difference
V. CONCLUSION Based on Probabilistic Histogram”, Procedia Computer Science, Vol.
86, pp. 265-268, 2016.
The process of facial recognition with the Haar Cascade [7] V. Jain, D. Patel, “A GPU Based Implementation of Robust Face
and Eigenface method is able to detect and recognize the face Detection System”, Procedia Computer Science, Vol. 87, pp. 156-163,
both during the day and night (with good light) as shown in the 2016.
test results. Although the type of detection is for straight faces [8] V.E. Machaca Arceda, K.M. Fernández Fabián, P.C. Laguna Laura, J.J.
(frontal faces), it is still able to detect the face when it is facing Rivera Tito, J.C. Gutiérrez Cáceres, “Fast Face Detection in Violent
to the side until about 15o (degrees). Video Scenes”, Electronic Notes in Theoretical Computer Science, Vol.
329, pp. 5-26, 2016.
The facial recognition process with the Haar Cascade and [9] P. Irgens, C. Bader, T. Lé, D. Saxena, C. Ababei, “An efficient and cost
Eigenface method is able to optimize facial recognition with effective FPGA based implementation of the Viola-Jones face detection
more than one face with accuracy up to 91, 67%. algorithm”, HardwareX, Vol. 1, pp. 68-75, 2017.
[10] N. Kazanskiy, V. Protsenko, P. Serafimovich, “ Performance analysis of
real-time face detection system based on stream data mining
frameworks”, Procedia Engineering, Vol. 201, pp. 806-816, 2017.