Vol-8 Issue-4 2022 IJARIIE-ISSN(O)-2395-4396

Improved LBPH using image processing and

Viola-Jones algorithm
Yash Uday Kulkarni
1
MCA 2 nd year, Department of computer science, Veermata Jijabai technological institute, Maharashtra,
India

Dr. Surekha Kohle.

Department of computer science, Veermata Jijabai Technological institute, Maharashtra, India

ABSTRACT
Face Recognition is a computer application that is capable of detecting, tracking, identifying or verifying human
faces from an image or video captured using a digital camera. Although lot of progress has been made in domain
of face detection and recognition for security, identification and attendance purpose, but still there a re issues
hindering the progress to reach or surpass human level accuracy. These issues are variations in human facial
appearance such as; difference in illumination, lighting condition, noisy facial images, scale, pose etc. In this paper
we introduce some advanced image processing techniques like contrast adjustment, image filtering, histogram
equalization etc. In addition to these some image blending functions can also be used to create a more robust model
for recognition. Apart from that, the rest of the implementation of the LBP process remains the same. At the end, we
also give a suggested improvement based on median filtering, which uses the median metric instead of the normal
LBP operator to improve the rate of recognition of the algorithm.
Keyword: -

1. Introduction-1

The Human face is a complicated ordeal to identify. The human brain is hardwired to understand and recognize
faces through years of evolution but the same cannot be said about a machine. Over the years, many algorithms have
been developed to recognize human faces. These models have been developed using AI, computer vision and other
machine learning models.

LBPH by all means, is an old algorithm by now. It was introduced in 1980 and since then has been used in many
applications. The main challenges for successful face detection and recognition systems are; illumination conditions,
scale, occlusion, pose, background, expression etc. Various algorithms and methods have been proposed to address
these challenges; N.Pattabhi Ramaiah uses illumination Invariant Face Recognition using Convolutional Neural
Networks to address illumination conditions, Abass addresses the issues of shift and rotation using complex wavelet
transform (CWT) and Fisherface. To address issues related to pose, Kish or et al Ref. [14] proposes robust pose
invariant face recognition using Dual Cross Pattern (DCP), LBP and Support Vector Machine (SVM).

This study has been divided into two main parts: In the first part we try to improve the accuracy and performance of
the algorithm using some advanced image pre-processing, combined with a powerful face detection algorithm called
the Viola-Jones algorithm. In the second part, we suggest a few changes in the original LBP operator itself, but as
stated earlier this study does contain the evidence to back the same. Thus, the improvement is proposed as a form of
further reading/study. The mathematics of the same do look promising, which is something to be noted.
.

17986 www.ijariie.com 1902

Vol-8 Issue-4 2022 IJARIIE-ISSN(O)-2395-4396

2. Related work
Kohonen Ref. is one of the early pioneers of the most famous face recognition system, which employed a simple
neural net using network of Eigenfaces by approximating eigenvectors through face images autocorrelation matrix.
Although, the method was not very successful to be practically implemented in a real-life environment due to
associated high demand for normalization and positioning when run in a large database with many types of face
conditions.

Better and more novel approaches than Kohonen approach for facial recognition using; Principal Component
Analysis (PCA), Fisherfaces and the traditional Local Binary Patterns (LBP) were proposed, particularly the LBP,
because it has a simple theory with computational simplicity, invariant with respect to any monotonic transformation
of gray scale, has powerful rotation-invariant analysis with a uniform pattern and discriminates excellently between
different various kinds of texture, but it is known that the LBP is not as robust as the viola-jones and other
algorithms for face detection, because of issues such as noise, illumination variation, background, pose, scale and
occlusion etc. K.

In another approach, facial recognition was achieved using Modified Lo cal Binary Pattern and Random Forest,
which the sign and magnitude features are combined for the improvement of facial texture classification
performance and when compared with the traditional LBP for multiple patch variations on a challenging facial
dataset, this method proven to be more accurate. All these techniques, methods, algorithms reviewed does not
wholly addresses issues affecting facial recognition accuracy such as illumination variation, noise, scale, sharp, pose
in one shot, while our method was able to do that and also our method focuses on enhancing features of input and
training images, thus improved LBP codes and achieved better recognition results.

3. Improved LBP algorithm

When studying the LBP algorithm, we noted that the performance dep ends upon the accuracy of the feature
extraction stage, which in turn depends upon the quality of images used by the application. In this study, we exploit
the same property of this algorithm. We apply some advanced image processing techniques to the input images to
remove any noise, blurs and errors introduced in the images due to the camera quality, illumination, shadows etc. Of
course, every such factor cannot be removed to create a perfect image, but it can be reduced reasonably.

Fig. Original LBP operator

But even before making some additions to enhance the algorithm, we choose a better face detection algorithm. In
this study, we make use of the Viola-Jones face detection algorithm.

3.1 Viola-Jones algorithm

17986 www.ijariie.com 1903

Vol-8 Issue-4 2022 IJARIIE-ISSN(O)-2395-4396

Developed in 2001 by Paul Viola and Michael Jones, the Viola-Jones algorithm is an object-recognition framework
that allows the detection of image features in real-time.

Despite being an outdated framework, Viola-Jones is quite powerful and its application has proven to be
exceptionally notable in real-time face detection.

Fig –2 The 3 Haar like features

There are 2 stages in the Viola-Jones Algorithm:

1.Detection
2.Training

1. Detection:
Viola-Jones was designed for frontal faces, so it is able to detect frontal the best rather than faces looking sideways,
upwards or downwards. Before detecting a face, the image is converted into grayscale, since it is easier to work with
and there’s lesser data to process. The Viola-Jones algorithm first detects the face on the grayscale image and then
finds the location on the colored image.

2. Training
The detected faces are stored in form of a database and a model is trained based on it. In our example, a real-time
face detection application has been developed to measure the accuracy of the alg orithm.
The application would start by detecting the face which is in front of the webcam of said machine. Then it would
take 64 images of the person and store it in a folder. This way, the algorithm would extract features from the face.

17986 www.ijariie.com 1904

Vol-8 Issue-4 2022 IJARIIE-ISSN(O)-2395-4396

Fig. 3 The face dataset created using Viola-Jones

3.2 Proposed Methodolog y

So, the methodology proposed here starts by replacing the default LBP face detection, in the LBPH algorithm, by
the above-mentioned Viola-Jones algorithm. To make use of the Viola-Jones algorithm, we can either make our own
Haar cascade file, or download a suitable one from the internet. A Haar cascade file is an XML file which contains
the facial vectors, or ‘Haar like features’ of a standard human face, which is used to detect face in a camera feed
using the front camera and the OpenCV library in python. This simple change alone has improved the rate of
detection of the LBPH algorithm, but the rate of recognition of faces remains largely unchanged, showing a minor
increase from 23% to 27%. The LBPH algorithm is fairly accurate but the rate of recognizing faces is a prevalent
problem. Though this paper does not counter the rate of recognition issue, it does propose a way, atleast in theory to
improve the same.

We start off by applying some contrast adjustment. Different values for alpha and beta, which are also known as the
gain and bias parameters, and the best outcome was achieved by alpha = 1.5 and beta = 0.0.

The equation for contrast adjustment which was used is

g(i,j)=α⋅ f(i,j)+β
Where β = 0.0 and α = 1.5

The next step we take is adding a filter to remove noise from the image. We say an image is noisy if the image is
grainy. In our study, a factor of 3 was used for median filtering.

17986 www.ijariie.com 1905

Vol-8 Issue-4 2022 IJARIIE-ISSN(O)-2395-4396

Fig.Noisy image Fig. Filtered(median)

Fig. Contrast adjusted Fig. Histogram equalization

Finally, we use histogram equalization to further improving the image quality and addressing the issue of global
lighting.
The equation for the same is
Eq = H’(CF(x,y))
Here CF(x,y) is the combined equation of the earlier two equations for contrast adjustment and median
filtering.

Methods of face Accuracy Incorrect

detection detections

LBP 91% 15

Haar 92% 12

Viola-Jones 96% 8

Table 1. Face detection comparison.

Table 1 shows that for real-time systems, like in our case Viola-jones algorithm has given the best performance
when compared to LBP face detection or the Haar-like faces detection. Fisherfaces is another similar algorithm but
here Viola-jones was used as it gave us the best results and is easy to understand.

17986 www.ijariie.com 1906

Vol-8 Issue-4 2022 IJARIIE-ISSN(O)-2395-4396

Methods of face Accuracy

recognition
LBP+SVM+PSO 96.54%

Original LBP 89.3%

DCP+LBP+SVM 97.5%

Proposed method (VJ + 98%

ILBPH)
Table 2. Comparison of various recognition algorithms

17986 www.ijariie.com 1907

Vol-8 Issue-4 2022 IJARIIE-ISSN(O)-2395-4396

17986 www.ijariie.com 1908

Vol-8 Issue-4 2022 IJARIIE-ISSN(O)-2395-4396

3.3 A suggested improvement in the original LBP operator

The improvement which is suggested in this section is based on theoretical research and needs further study. The
experiment and testing for this improvement is out of the scope of this paper and thus no experimental evidence is
provided.
While working on the various filters to be used in image processing, we went through different methods like
Gaussian, Bilateral, median and mean filters etc. Medina filter was of particular interest.
As mentioned earlier, the LBPH is a very accurate face recognition algorithm. But what has made it somewhat
unsuitable these days is the lower rate of face recognition. In our tests, we found out the same issue in our real-time
application that out of 100, 98 times the face recognized was correct, but the recognition needed some amount of
time and multiple attempts. This might work on a small-scale project like this, but in large scale applications, like
security feeds for example, this makes the algorithm highly undesirable. W e found out that the primary reason for
this is illumination changes, expression changes and also the altitude of the face on the screen. To counter this
problem, we propose modified LBP operator.

17986 www.ijariie.com 1909

Vol-8 Issue-4 2022 IJARIIE-ISSN(O)-2395-4396

Fig. The original LBP operator.

Fig. The Median LBP operator

The MLBPH operator inspired from median filtering, replaces the central grey value (pixel value) of a
given matrix that is, a part of any given image, with the median of all the values present in the matrix. The
rest of the process is the same as the local binary patterns' algorithm, where a histogram is generated for
each image and then face recognition is done based on the similarity of histograms of the testing image
and the trained data. Using the median, in theory should give a better approximation of the part of the
picture, especially when the illumination is a problem in said image. We could not provide an
experimental proof for the said improvement and hence no conclusive proof has been given. The results
do seem to provide an improvement in certain conditions in theory, but it might not always translate to
better performance in real-life applications. That being said, this can be used as an inspiration for furthur
study in the same topic.
The principle used in the logic of this improvement is based on measures of central tendency. It is a fact
that for a highly distributed data (here pixels), where the values are very distinct, in such a case median
would be a better measure of central tendency as compared to others. This is especially true in images
where the illumination is bad. The pixel values of such images are greatly varied. In such cases, using
median instead of the default center pixel value might yield better results.

4. CONCLUSIONS
In conclusion, in this study we apply some advanced image processing techniques to the input images, like contrast
adjustment, filtering, and histogram equalization. This simple addition has improved the accuracy of algorithm from
89% to 97%. Also, we use the Viola-Jones algorithm instead of the LBP face detection, which improved the rate of
detection. Lastly, to improve the rate of recognition of the original LBP algorithm, we have suggested a change
inspired from median filtering. This change tackles the global illumination problem but does not tackle the altitude
and expression change problem. The last to drawbacks of the LBP algorithm can be used as a further study, which
can help resolve the said issues.

17986 www.ijariie.com 1910

Vol-8 Issue-4 2022 IJARIIE-ISSN(O)-2395-4396

5. REFERENCES

[1] Nisha Maitreyee Dutta. Improving the recognition of faces using LBP and SVM
optimized by PSO technique. Int J Exp Diabetes Res 2017;5(4):297–303. ISSN:
2321-9939
[2] Priya TS Vishnu, Vinitha Sanchez G, Raajan NR. Facial recognition system using
local binary patterns (LBP). Int J Pure Appl Math 2018;119(15):1895– 9.
[3]. Bhangale Kishor B, Jadhav Kamal M, Shirke Yogesh R. Robust pose invariant face
recognition using DCP and LB. International Journal of Management, Technology
and Engineering September 2018;8(IX):1026–34. ISSN NO: 2249-7455
[4]. M. Joe Minichino, Joseph Howse. Learning OpenCV 3 Computer Vision with Python. 2016: 16-82.

17986 www.ijariie.com 1911