IEEE-International Conference on Recent Trends in Information Technology, ICRTIT 2011

MIT, Anna University, Chennai. June 3-5, 2011

LOCAL BINARY PATTERNS AND ITS

VARIANTS FOR FACE RECOGNITION
K.Meena#1, Dr.A.Suruliandi*2
#
Assistant Professor, Department of Computer Science and Engineering, Sardar Raja college Of Engineering, Tirunelveli,
India.
*
Associate Professor, Department of computer Science and Engineering, Manonmaniam Sundaranar University, Tirunelveli,
India.
1
[email protected]
2
[email protected]

Abstract—Face recognition is one of the most important tasks in access control, Computer logon, Surveillance camera,
computer vision and Biometrics. Texture is an important spatial Criminal identification and ATM.
feature useful for identifying objects or regions of interest in an Face recognition system can be grouped as 1.structure
image. Texture based face recognition is widely used in many based 2.appearance based. In structure based method [12] a
applications. LBP method is most successful for face recognition.
set of geometric face features, such as eyes, nose, mouth
It is based on characterizing the local image texture by local
texture patterns. In this paper performance evaluation of Local corners, is extracted, the position of the different facial
Binary Pattern (LBP) and its modified models Multivariate features form a feature vector as the input to a structural
Local Binary Pattern (MLBP), Center Symmetric Local Binary classifier to identify the subject. In the second method [2], the
Pattern (CS-LBP) and Local Binary Pattern Variance (LBPV) appearance of face as input to decision making and they can e
are investigated. Facial features are extracted and compared further categorized as holistic and component based. The
using K nearest neighbour classification algorithm. G-statistics holistic appearance methods operate on the global properties
distance measure is used for classification. Experiments were of face image. Nowadays ,appearance based methods not only
conducted on JAFFE female, CMU-PIE and FRGC version2 operate on the raw image space ,but also other spaces ,such as
databases. The results shows that CS-LBP consistently performs
wavelet ,local binary pattern and ordinal pattern spaces.
much better than the remaining other models.
The Local Binary Pattern is originally proposed by
Ojala [7] for the aim of texture classification, and then
Keywords—Face recognition, local binary pattern (LBP),
extended for various fields, including face recognition [9],
Multivariate Local Binary Pattern (MLBP), Center Symmetric
Local Binary Pattern (CS-LBP), Local Binary Pattern Variance face detection [3], facial expression recognition [13].The
(LBPV). Local Binary Pattern is a non parametric operator which is
used for describing a local spatial structure of an image. The
I. INTRODUCTION Local Binary Patter method is computationally simple and
Facial recognition plays a vital rule in human computer rotation invariant method for face recognition [9].Adaptive
interaction [4]. A Face recognition system can be either smoothing for face image normalization under variation of
verification or an identification system depending on the illumination is presented by Y.K.Park [8]. The illumination is
context of an application. The verification system estimated by iteratively convolving the input image with a 3-
authenticates a person’s identity by comparing the captured by-3 averaging kernel weighted by a simple measure of the
image with his/her own templates stored in the system. It illumination discontinuity at each pixel. In particular, weights
performs a one to one comparison to determine whether the of a kernel are encoded into a local binary pattern (LBP) to
person presenting himself/herself to the system is the person achieve fast and memory efficient processing.
he/she claims to be. An identification system recognizes a Face image is divided into several regions and LBP
person by checking the entire template database for a match. It is applied and features are extracted over the region. These
involves a one to many searches. The system will either make features are concatenated to form face descriptor [10].
a match or subsequently identify the person or it will fail to Although face recognition with local binary pattern has been
make a match. proven to be a robust algorithm, it suffers from heavy
The human ability to recognize face is remarkable. We computational load due to the very high dimensional feature
can recognize the thousands of faces learned throughout our vectors that are extracted by concatenating the LBP
lifetime and identify familiar faces at a glance even after years histograms from each local region. A new multichannel filter
of separation. This skill quite robust, despite large changes in based Gabor wavelet is designed based on theory and
the visual stimulus due to viewing practicality. Its center frequency is the range from low
conditions ,expressions ,aging and distractions such as glasses frequency to high frequency, its orientation is 6 and scale is 6.
or changes in hairstyle or facial hair. Existing biometric It can extract the feature of low quality facial expression
systems are developed for corporate user applications like image target, and have well robust for automatic facial
expression recognition [5].

978-1-4577-0590-8/11/$26.00 ©2011 IEEE 782

Authorized licensed use limited to: Consortium - Algeria (CERIST). Downloaded on February 16,2024 at 22:19:47 UTC from IEEE Xplore. Restrictions apply.
 IEEE-ICRTIT 2011

MLBP is proposed by Arco Lucifer [1] for texture the radius and p is the number of neighborhood points on the
segmentation. Most of the images are multi band in nature. So circle .From Fig.2 we can write,
this method is widely used for image classification and p −1
­1 if x ≥ 0
segmentation. CS- LBP method was introduced by Marko
Heikkila [6]. This new descriptor has several advantages such
LBPp , r = ¦ s( g − g ) 2
p=0
i c
i
, S(x) = ®
¯0 otherwise
(1)

as tolerance to illumination changes, robustness on flat image

areas and computational efficiency. LBP variance (LBPV) is The concept of uniform patterns is introduced to reduce
proposed by Zhenhua Guo [14] to characterize the local the number of possible bins. Any LBP pattern is called as
contrast information into one dimensional LBP histogram. In uniform if the binary pattern consists of at most two bitwise
this paper LBP and its variants methods are evaluated in transitions from 0 to 1 or vice versa. For example if the bit
JAFFE female database for face recognition. Among these pattern 11111111(no transition) or 00110000 (two transitions)
methods, the best method will be tested by CMU PIE, FRGC are uniform where as 10101011 (six transition) are not
version2 databases. uniform. The uniform pattern constraint reduces the number
The rest of the paper is organized as follows. Section II of LBP pattern from 256 to 58 and it is very useful for face
reviews about LBP and its derivatives MLBP, CS-LBP and detection [10].
LBPV. Section III explains about classification principle.
Section IV reports the experimental data and the results on B. Multivariate Local binary pattern(MLBP)
JAFFE female, CMU-PIE and FRGC version2 databases. The Multivariate Local Binary Pattern operator, MLBP
Section V gives the conclusion of this paper. c was developed by Arco Lucifer [1] which describes local
pixel relations in three bands. In addition to the spatial
II. TEXTURE MODELS interactions of pixels within one band, interactions between
bands are considered. Thus, the neighborhood set for a pixel
consist the local neighbours in all three bands (Fig 3).
A. Local binary pattern(LBP)
Local Binary Pattern was introduced by Timo ojala [11].
The standard version of the LBP of a pixel is formed by
thresholding the 3X3 neighborhood of each pixel value with
the center pixel’s value. Let gc be the center pixel gray level
and gi (i=0,1,..7) be the gray level of each surrounding pixel.
Fig.1 illustrate the basic LBP operation. If gi is smaller than
gc , the binary result of the pixel is set to 0 otherwise set to 1.
All the results are combined to get 8 bit value. The decimal
value of the binary is the LBP feature.

Fig. 3.MLBP texture measure describes spatial relations within a band and
between bands

sign(gi b1 - gc b1) + sign(gi b2 - gc b1) +

Fig. 1 Illustration of Basic LBP operator
sign(gib3 - gc b1) + sign(gi b1 - gc b2) +
p−1
MLBP= ¦sign(g
i=0
i
b2
- gc b2) + sign(gi b3 - gc b2) + (2)

sign(gib1 - gc b3) + sign(gi b2 - gc b3) +

sign(gi b3 - gc b3)
From Eqn.2 the local threshold is taken from these bands,
Fig.2. The LBP operator of a pixel’s circular neighborhoods with r=1 ,p=8 which makes up a total of nine different combinations. This
Bilinear interpolation method is used for a sampling point results in the following operator for a local color texture
does not fall in the center of the pixel. Let LBPp,r denote the description. The color texture measure is the histogram of
LBP feature of a pixel ‘s circularly neighborhoods, where r is MLBP c occurrence, computed over an image or a region of an

783
Authorized licensed use limited to: Consortium - Algeria (CERIST). Downloaded on February 16,2024 at 22:19:47 UTC from IEEE Xplore. Restrictions apply.
 Local Binary Patterns and its Variants for Face Recognition

image. This single distribution contains P ×32bins (e.g. P =8 III. CLASSIFICATION PRINCIPLE
results in 72 bins).
A. Training
C. Center Symmetric Local binary pattern(CS-LBP) In the training phase, the texture features are extracted from
The CS-LBP is another modified version of LBP. It model the samples selected randomly belonging to each face class,
was developed by Marko Heikkila [6] for the recognition of using the proposed feature extraction algorithm. The average
object in PASCAL database. The original LBP was very long of these features for each face class is stored in the feature
its feature is not robust on flat images. In this method, instead library, which is further used for classification.
of comparing the gray level value of each pixel with the center
pixel, the center symmetric pairs of pixels are compared B. Texture Similarity
(Fig.4). CS-LBP is closely related to gradient operator. It To find out the similarity between training models and
considers the grey level differences between pairs of opposite testing sample G-statistic distance measure is used. Similarity
pixels in a neighborhood. So CS-LBP take advantage of both between the textures is evaluated by comparing their pattern
LBP and gradient based features. It also captures the edges spectrum. The spectrums histograms are compared as a test of
and the salient textures. goodness-of-fit using a non-parametric statistics, also known
as the G-statistics [7].The G statistic compares the two bins of
two histogram and is defined as

ªª n º º
« « ¦ ¦ f i log f i » »
« ¬ s , m i =1 ¼ »
« n »
« − ª ¦ §¨ ¦ f ·¸ log § n ·º »
« ¨ ¦ fi ¸»
« ¬ s , m © i =1 i ¹ © i =1 ¹¼ »
Fig. 4 CS-LBP feature for a neighbourhood of 8 pixel G = 2« » (6 )
« ª n § · § ·º »
The CS-LBP features can be computed by « − « ¦ ¨ ¦ f i ¸ log ¨¨ ¦ f i ¸¸ » »
N / 2−1 « ¬« i = 1 © s . m ¹ © s ,m ¹ ¼» »
­1 if x ≥ t
CS − LBPp,r,t = ¦ s( g − g (N / 2) ) 2 , s(x)= ®¯0 otherwise
i i
i
(3) «
« + ª §¨ n · § n ·º »
»
« ¨ s , m i = 1 f i ¸¸ log
¦ ¦ ¨¨ ¦ ¦ f i ¸¸ »
i =0
« »
¬ «¬ © ¹ © s , m i =1 ¹ »¼ ¼
Where gi and gi+n/2 correspond to the gray level of center
symmetric pairs of pixels (N in total) equally spaced on a
circle of radius r. It also reduces the computational complexity C. Classification
when compared with basic LBP [6]. In the texture classification phase, the texture features are
extracted from the test sample x using the proposed feature
D. Local binary pattern variance(LBPV) extraction algorithm, and then compared with model feature
The LBPV descriptor proposed by Zhenhua [14] offers a using K-Nearest Neighbor classification algorithm .In
better result than LBP. Local invariant features have the experiment 1, K=1 is used. (ie) minimum distance classifier is
drawback of losing global spatial information, while global used. Minimum distance between the model feature value and
features preserve little local texture information. LBPV the sample feature value is calculated.
proposes an alternative hybrid scheme; globally rotation
invariant matching with locally variant LBP texture features. IV. EXPERIMENTS
It is a simplified but efficient joint LBP and contrast A. Experimental Data
distribution method. LBPp,r /VARp,r is powerful because it
exploits the complementary information of spatial pattern The discrimination capability of any method is done by
and local contrast. Threshold values are used to quantize the experimental tests using the bench mark data base.
VAR of the test images computed to partition the total
distribution into N bins with an equal number of entries.
N M
LBPV P ,R (k ) = ¦ ¦ W ( LBP
i =1 j =1
P, R ( i , j ), k ) k ∈ [ 0 , K ] (4)

where
­ var P, R ( i , j ) , LBP P, R (i, j ) = k ½ Fig 5 Sample Images from JAFFE Female database
W ( LBP P, R ( i , j ), k ) = ® ¾ (5)
¯0 , otherwise ¿
Fig. 5 shows the images from JAFFE female database. It
These threshold values are used to quantize the variance of contains 56 images. (8 images X 7 poses=56 images).All the
test images.

784
Authorized licensed use limited to: Consortium - Algeria (CERIST). Downloaded on February 16,2024 at 22:19:47 UTC from IEEE Xplore. Restrictions apply.
 IEEE-ICRTIT 2011

training and testing images are pre-processed to the size of the increase in window size as well as the increase in the
120X120. number of samples taken for classification. Our experimental
results show that CS-LBP provides better results than the
remaining other methods.
TABLE 2 RECOGNITION RATE FOR DIFFERENT NUMBER OF
SAMPLES

Window size=30X30
Fig 6. Samples from the CMU-PIE face database. The first image from the
left is a sample training image and the others are the sample testing images.

Fig.6 and Fig.7 represents the input images for Fig.6

represents the images from CMU-PIE database. Among these Recognition Rate (%)
five images only one is used for training and the remaining
four images are used for testing purpose. SL No of
NO Samples CS-
LBP MLBP LBPV
LBP

1 10 15 35 20 21.6
Fig 7. Samples from the FRGC Version2 face database. The first image from
the left is a sample training image and the others are the sample testing
images.

Similarly Fig 7 shows the images from FRGC version2 2 20 39 42 50 47

database. Here first image from the left side is used for
training and the remaining images are used for testing phase.
B. Experimental comparisons on JAFFE Female database 3 30 82 84 86 82

TABLE 1 RECOGNITION RATE FOR DIFFERENT WINDOW SIZE

No of testing Samples=30 C. Experimental comparisons on CMU-PIE and FRGC

Version2 databases for illumination variations
Recognition Rate (%) Experiment#3 is conducted on CMU-PIE and FRGC
version2 database which is shown in Fig.6. The first image
SL Window from the left side is taken as training image and the remaining
NO Size LBP MLBP CS-LBP LBPV four images are used as testing images. In training phase,
facial features are extracted by CS-LBP method and stored in
the database.

1 10x10 15 25 24 20

80
2 20x20 34 35 43 40.5 70
60
50
3 30x30 81 83 87 84 40
CMU-PIE
30
20
Experiments are conducted on JAFFFE database by varying 10
the window size and also varying the number of input samples 0
from each image. During experiment#1 the number of training 10x10 20x20 30x30
sample is fixed as 10 and the window size is varied from
10X10. During experiment#2, the window size is fixed as
30X30 and the number of testing sample is increased from 10.
Table 1 and Table 2 shows that recognition rate increases with

785
Authorized licensed use limited to: Consortium - Algeria (CERIST). Downloaded on February 16,2024 at 22:19:47 UTC from IEEE Xplore. Restrictions apply.
 Local Binary Patterns and its Variants for Face Recognition

During testing phase, facial features are extracted by using [5] M.Lyons,S.Akamatsu,etc.,”Coding Facial Expressions with Gabor
wavelets “,Proceeding of the third IEEE International conference on
the above method and the difference between two facial
Automatic face and Gensture Recognition,Nero Japan,(1998)200-205.
features is evaluated by G-statistic distance measure with [6] Marko Heikkila,Matti Pietikainen, Cordelia Schmid,”Description of
k=1(nearest neighbour classification) algorithm. This interest regions with Local BinaryPattern”, June 2008.
experimental results show that face recognition is mainly [7] T.Ojala,M.Pietikainan,and D.Hawood,” A Comparative study of
texture images with classification based on featured
depends on illumination changes.
distributions’,pattern recognition,29(1):51-59,1996.
TABLE 4 RECOGNITION RATE OF FRGC VERSION2 DATABASE BY [8] Y.K.Park and J.K.Kim ,”Fast adaptive smoothing based on LBP for
CS-LBP METHOD face recognition”,ELECTRON LETTERS, 22nd November 2007,Vol
43,No 24.
[9] Timo ojala , Matti Pietikäinen, Topi Mäenpää,” Multiresolution gray
scale and rotation invariant Texture Classification with Local Binary
90 Patterns” IEEE Transactions on pattern analysis and Machine
80 Intelligence Volume 24 Issue 7, July 2002.
70 [10] Timo Ahonen,Matti Pietikainen,” Face Description with Local Binary
Patterns: Application to Face Recognition”, IEEE Transactions on
60 pattern analysis and Machine Intelligence Volume 28 ,No
50 12,DECEMBER 2006.
FRGC [11] Timo Ahonen,Matti Pietikainen and T.Maenpaa,” Multiresulation gray
40 scale and rotation invariant texture classification with local binary
Version2
30 pattern”, IEEE Transactions on pattern analysis and Machine
Intelligence Volume 24 ,No 7,pp 971-987,2002.
20 [12] M.Turk and A.Pentland ,”Eigen faces for face recognition”,J.Cognitive
10 Neuroscience,Vol 3,no 1,pp.71-86,1991.
0 [13] Zhao and M.Pietikainan,Dynamic,” texture recognition using Local
Binary Patterns with an application to facial expressions”.IEEE
10x10 20x20 30x30 Transaction.Pattern Anal.Mach.Intell.,29(6):915-928,2007.
[14] Zhenhua Guo,Lei Zhang,David Zhang,”Rotation Invariant texture
classification using LBP variance (LBPV) with global matching”.
Pattern Recognition 43 (2010) 706–719.

TABLE 3 and 4 shows the recognition rate vs window

size of CS-LBP method on CMU-PIE and FRGC Version2
databases. It shows that recognition rate increases with
increase in window size. CMU-PIE database gives better
results than FRGC Version2 database under different lighting
conditions.

V. CONCLUSIONS
LBP is grey scale invariant and rotational invariant. This
property is well suitable for many applications. The facial
recognition based on Local Binary Patterns is extremely
simple. In this paper LBP and its modified models CS-LBP,
MLBP and LBPV were analysed. CS-LBP performs very well
and gives the recognition rate of 87% with the JAFFE female
database. CMU-PIE and FRGC Version2 databases are
experimented by the same model under different illuminations.
The model gives the recognition rate of 80% for CMU-PIE
and 82% for FRGC Version2 database. CS-LBP provide good
recognition rate than other methods and also it consumes less
computational time.

REFERENCES
[1] Arko Lucieer, Alfred Stein and Peter Fisher ,” Multivariate Texture-
based Segmentation of Remotely Sensed Imagery for Extraction of
Objects and Their Uncertainty”.
[2] K.Etemad and R.Chellappa,” Discriminant Analysis for face
recognition of Human Face images,” I.Optical Soc.
Am.,Vol.14,PP.1724-1733,1997.
[3] A.Hadid, ,M.Pietikainan, Timo Ahonen,”A Discriminative Facial
feature for detecting and recognizing faces,In CPVR(2),pages 784-
804,2004.
[4] Handbook of Face Recognition,S.Z Li and A.K.Jain,eds.Springer,2005.

786
Authorized licensed use limited to: Consortium - Algeria (CERIST). Downloaded on February 16,2024 at 22:19:47 UTC from IEEE Xplore. Restrictions apply.