Computer Methods and Programs in Biomedicine 197 (2020) 105622

Contents lists available at ScienceDirect

Computer Methods and Programs in Biomedicine

journal homepage: www.elsevier.com/locate/cmpb

Parallel ensemble learning of convolutional neural networks and local

binary patterns for face recognition
Jialin Tang a,b, Qinglang Su b, Binghua Su a, Simon Fong c,∗, Wei Cao a, Xueyuan Gong a,∗
a
Beijing Institute of Technology, Zhuhai 519088, China
b
City University of Macau, Macau, China
c
Department of Computer and Information Science, University of Macau, Macau, China

a r t i c l e i n f o a b s t r a c t

Article history: Background and Objective: Face recognition success rate is influenced by illumination, expression, posture
Received 28 April 2020 change, and other factors, which is due to the low generalization ability of a single convolutional neural
Accepted 15 June 2020
network. A new face recognition method based on parallel ensemble learning of convolutional neural
networks (CNN) and local binary patterns (LBP) is proposed to solve this problem. It also helps to improve
Keywords: the low pedestrian detection rate caused by occlusion.
Convolutional Neural Networks (CNN)
Methods: First, the LBP operator is employed to extract features of the face texture. After that, 10 convo-
Local Binary Patterns (LBP)
Ensemble learning lutional neural networks with 5 different network structures are adopted to further extract features for
Face recognition training, to improve the network parameters and get classification result by using the Softmax function
after the layer is fully connected. Finally, the method of parallel ensemble learning is used to generate
the final result of face recognition using majority voting.
Results: By this method, the recognition rates in the ORL and Yale-B face datasets increase to 100% and
97.51%, respectively. In the experiments, the proposed approach is illustrated not only enhances its toler-
ance to illumination, expression, and posture but also improves the accuracy of face recognition and the
poor generalization performance of the model, which is normally caused by the learning algorithm being
trapped in a local minimum. Moreover, the proposed method is combined with a pedestrian detection
model as a hybrid model for improving the detection rate, which shows in the result that the detection
rate is improved by 11.2%.
Conclusion: In summary, the proposed approach greatly outperforms other competitive methods.
© 2020 Elsevier B.V. All rights reserved.

1. Introduction tional feature extraction approach is not an ideal face recognition

solution. Since Histogram of Oriented Gradient (HOG) [13] main-
Since face information processing has many advantages, such as tains good invariance to optical and geometric deformation of im-
high safety coefficient, convenient data acquisition and easy popu- age, and local binary patterns (LBP) [8] has the advantages of
larization in biometric feature recognition. Face recognition tech- grayscale invariance, insensitivity to illumination and so on, they
nology is useful in many finance and security applications, such as are employed to get facial features for face recognition.
video surveillance and intelligent payment. It is one of the most In this era, a large number of face training data sets have
popular research directions in machine learning and computer vi- emerged, and a huge number of attentions has been attracted
sion at present. to deep learning. The neural network model can process two-
Face recognition is often used in unconstrained condition. The dimensional images, learn the characteristics of the images from a
collected images are affected by several environmental factors such large number of samples, and classify the two-dimensional images
as illumination, expression, posture and so on. Therefore, the tradi- according to the learned characteristics, so as to realize the face
recognition. The feature extraction by machines can avoid exces-
sive intervention of subjective factors. At the same time, convolu-
∗
Corresponding authors. tional neural network is widely adopted in face recognition appli-
E-mail addresses: [email protected] (J. Tang), [email protected] (Q. Su), cations because of its self-learning ability, parallel processing abil-
[email protected] (B. Su), [email protected] (S. Fong), [email protected] (W. ity, good fault tolerance and generalization performance.
Cao), [email protected] (X. Gong).

https://doi.org/10.1016/j.cmpb.2020.105622
0169-2607/© 2020 Elsevier B.V. All rights reserved.
2 J. Tang, Q. Su and B. Su et al. / Computer Methods and Programs in Biomedicine 197 (2020) 105622

The convolutional neural network can detect the facial features Therefore, the parallel ensemble learning based on LBP and
from image directly, but it will learn the noise of the image in the CNN is proposed in this paper to extract face image features, which
meantime. Both HOG and LBP can process the face images and re- reduces the depth of the convolutional neural network, improves
duce the noise interference, thus making the features of images the accuracy of classification and reduces the influence of illumi-
more obvious. Since HOG is more focused on extracting the pre- nation, expression and posture change, etc. The main difference of
sentational and shape features of the target, while LBP can extract our approach and the existing approaches are that our approach
the texture features of the target, therefore, LBP has better effect utilizes LBP to extract facial features firstly. Afterwards, they are
on extracting facial features. In this paper, LBP is employed to de- fed into ResNet and classified, where ResNet is a cutting-edge CNN
tect facial texture features to reduce the influence of illumination model for classifying images.
and expression; and CNN and skip connection are used for paral-
lel convolution processing, so as to reduce the training time and 3. FACE recognition based on parallel ensemble learning of LBP
improve the accuracy of classification. Meanwhile, by introducing and CNN
the parallel ensemble learning method and parallel connection of
two or more convolutional neural networks with different struc- This paper mainly studies the face recognition based on paral-
tures for face recognition, the diversity of the ensemble individu- lel ensemble learning of LBP and CNN. In this paper, LBP is first
als is improved and the generalization ability of the network is en- employed to analyze the texture of the input images, then CNN
hanced. One face recognition method based on parallel ensemble is utilized to get the facial features of the images processed by
learning of LBP and CNN is proposed in this paper, which effec- LBP, finally, the parallel ensemble learning method is utilized to
tively improves the face recognition accuracy. improve the poor generalization performance of the CNN caused
In the experiment part, two different experiments are con- by the learning algorithm being trapped in a local minimum, so as
ducted. First, ORL [8] and Yale-B [13] face data sets are adopted to to improve the effect of distinguishing different faces. The imple-
test the accuracy of our proposed CNN model in face recognition mentation flow of the proposed approach is given in Fig. 1, and the
problem. In detail, PCA, HOG-CNN, CNN and the proposed method detailed implementation way is described as follows.
are compared in the experiment, the result of which demonstrates
that our method outperforms the others. Second, our method for 4. Local Binary Pattern (LBP)
face recognition is combined with a CNN model for pedestrian de-
tection, we call it hybrid model. This hybrid model improves the The local binary pattern, with its simple principle, low compu-
accuracy of pedestrian detection caused by occlusion with our pro- tational complexity, grayscale invariance, and illumination insensi-
posed face recognition model. Intuitively, human face is a small tivity, can extract the texture features of images and fuse the over-
part of a whole human in an image. Therefore, even half of the all features of an image.
human is occluded, the pedestrian is available to be detected as In this paper, the LBP model improved by Ojala [8] and other
long as his/her face is clear. researchers can obtain the texture features of images by changing
the radius of circle and the number of pixels. Bilinear interpola-
2. Literature review tion is used to obtain the point gray value that is not in the center
of the pixel box, which makes the algorithm more robust. Using
Tahira et al. [1] used PCA to reduce face dimensions so as to the improved LBP, the radius and pixels are expressed by R and
realize face recognition. Face image acquisition is influenced by il- P, respectively. The LBP algorithm for different radii and pixels is
lumination, expression and posture, which results in a large differ- demonstrated in Fig. 2.
ence of the same individual, and the reduction of the face recog- The basic idea of LBP is to compare the gray value of every
nition rate [2]. The face recognition approach based on local bi- neighboring pixel with that of the center pixel. Taking the radius of
nary pattern introduced by Ahonen et al. [3] can divide a face im- 1 and the number of pixels of 8 as an example, taking the center
age into several regions for face recognition. The face recognition point as the base point, the gray value of the central point is com-
approach based on multi-direction local binary pattern proposed pared with the gray values of 8 pixels in neighborhood. If the gray
by Liu et al. [4] acquires the feature vector by replacing a single value of the neighboring pixels is greater than that of the center
pixel with the regional mean value, which not only introduces the pixel, the gray values of all neighboring pixels are set to 1; on the
whole spatial image information, but also reduces the dimensions contrary, the gray values of all neighboring pixels are set to 0. The
of the image. By this way, the face recognition rate under com- LBP-based image feature extraction procedure is shown in Fig. 3.
plex illuminations is significantly improved. The face recognition The formula of LBP can be expressed as:
method based on a hybrid model of CNN and LBP proposed by
Wang et al. [5] can effectively overcome the disadvantage of poor 
P−1
LBPP,R (x, y ) = 2n s(in − ix,y ) (1)
grayscale stability of CNN and reduce the influence of illumina-
n=0
tion, expression and posture change. The face recognition approach

based on a hybrid model of HOG and CNN proposed by Ahamed 1, x ≥ 0
et al. [6] makes face recognition by inputting CNN with the shape S (x ) = (2)
0, x < 0
of the target. There are still some problems in the above methods.
For example, with the change of illumination intensity and pos- In which, LBPP,R (x, y) represents the LBP texture feature with
ture, the recognition rate of PCA will be greatly reduced. The HOG the center pixel (x, y), the radius R, and the neighboring pixels P.in
is more focused on extracting the presentational and shape fea- represents the gray value of the nth neighboring pixel, and ix,y rep-
tures of the target, instead of specific facial feature points. A large resents the gray value of the center pixel.
number of experiments prove that the increasement of depth and The texture features after being extracted by LBP are insensitive
width of the network improves the accuracy. However, with the to light change, and they are less affected by illumination varia-
deepening of the convolutional neural network, problems like the tions than that of the original image. Facial features of the same in-
increasing of parameters and the surging of computations will oc- dividual with various illumination intensities are extracted by LBP
cur. A deeper network can lead to a rapid saturation of accuracy, and the effect is shown in Fig. 4. In addition, as shown in Fig. 4,
and after saturation, a higher error rate will occur as the number not only is LBP able to capture the facial features with only a little
of training times increases [7]. affection of illumination, also that LBP preserves adequate details
 J. Tang, Q. Su and B. Su et al. / Computer Methods and Programs in Biomedicine 197 (2020) 105622 3

Fig. 1. Flow Chart for the Implementation Method Introduced in the Paper.

Fig. 2. Schematic Diagram of the LBP Algorithm.

Fig. 4. Texture Features Extracted from Face Images.

famous Batch Normalization [10] for skip connection [7] via CNN.
The network structure in Fig. 5 is one of the CNN models used in
this paper, which consists of four convolutional layers, five maxi-
Fig. 3. Calculation Process with the LBP Algorithm. mum pooling layers and one stacking model of Inception modules.

5.1. Convolutional layer

of facial features (e.g. hair) in order to avoid misclassifying two dif-
ferent people as same ones just because of their same expressions. The convolutional layer is composed of one or more convo-
In other words, CNNs and other classifiers may be misguided by lutional feature maps, and each convolution surface is calculated
the same expressions between two different people without giving from the input, the convolution kernel and the activation function.
more detailed facial features. The input of convolutional layer is assumed to be a M × M matrix
In order to preserve more facial features for the CNNs, in this x, the size of convolution kernel is K × K matrix w, the bias is b,
paper, the texture feature images extracted by LBP are directly and the convolutional feature map is matrix h. Then, the convolu-
used as inputs to the CNNs for learning to improve the accuracy. tion formula is:
h = f (conv(x, w ) + b) (3)
5. Convolutional Neural Network (CNN)
In which, f( • ) is an activation function, conv( • ) is a convolu-
In the paper, after the texture extraction of LBP, the facial fea- tion.
tures are further extracted by CNN. In order to increase the net- The literature [10] shows that the receptive field of a series of
work width, accelerate the training time and improve the accuracy small convolution kernels is consistent with that of a large con-
of classification, this paper adopts Inception module [9] and the volution kernel, the network parameters can be reduced by de-
4 J. Tang, Q. Su and B. Su et al. / Computer Methods and Programs in Biomedicine 197 (2020) 105622

Fig. 6. Yale-B Dataset Samples.

Fig. 7. ORL Dataset Samples.

6. Classification and parallel ensemble learning

In this paper, several convolutional neural networks based on

Fig. 5. CNN Structure Diagram. different network structures of LBP are used to learn and classify
the training datasets. After training, the final results are outputted
by vote to construct a face classifier. When calculating the accu-
composing the convolution kernel [11], and the deeper network is racy of the test dataset, each convolutional neural network out-
able to improve the accuracy of the model. So, reducing the con- puts a classification result for each face image in the test dataset.
volution kernel size and decomposing the convolution kernel can The final classification result is obtained by majority vote, which is
make the network structure deeper, the computation amount re- mathematically expressed as follows:
duced and the accuracy improved. Therefore, the maximum size of o = Gmax (o1 , o2 , o3 , · · · , on ) (7)
the convolutional kernel employed in the paper is 5 × 5, and 3 × 1
and 1 × 3 convolution kernels are used to replace the 3 × 3 con- In which, o is the final output result,o1 ,o2 ,o3 ,, on are the out-
volution kernel in order to reduce the parameters and calculation put results of n convolutional neural networks, Gmax is a function
amount, and improve the accuracy. of the output result with the most votes.

7. Experimental results and analysis

5.2. Pooling layer
7.1. Experimental dataset and environment
Pooling layer is also called subsampled layer. The input of the
pooling layer can be either the output of a convolutional layer or For the purpose of verifying the effectiveness of the proposed
that of a pooling layer. The pooling layer can capture features and approach, two public face image datasets, Yale-B and ORL, are se-
reduce the size of the input image, and it is normally put after the lected in this paper. The Yale-B dataset consists of 38 individuals
convolutional layer. The pooling layer includes several types, such with 576 face images per person in 9 postures and 64 illumina-
as Max Pooling and Average Pooling. These two types of pooling tion conditions, while the ORL dataset consists of 40 individuals
layers are mathematically expressed as follows: with 10 face images per person. In addition, CBCL [12] and INRIA
[13] pedestrian datasets are selected to test the effectiveness of our
max (P )
F = Dl,l (4) proposed method to improve the detection rate of pedestrian.
The face images of these two datasets contain the changes in
illumination, posture and expression. In comparison, the illumina-
avg ( P )
F = Dl,l (5) tion variation of Yale-B is larger than that of ORL, while posture
and expression variations of ORL are larger than that of Yale-B. In
In which, P is the input vector, Dl,lmax is the maximum pooling this experiment, the two datasets are separated into two datasets,
of the l × l block size, Dl,l
avg is the average pooling of the l × l block respectively, that is, training set and test set. For each face cate-
size, and F is the output pooling face. gory in the Yale-B dataset, the first 32 images are selected and put
into the training subset and the last 32 images into the test sub-
set. For each face category in the ORL dataset, the first 5 images
5.3. Fully-connected layer
are selected and put into the training subset and the last 5 images
into the test subset. Some face data samples are shown in Figs. 6
In the convolutional neural network, a fully-connected layer is
and 7.
often added after all convolutional and pooling layers in order to
The main performance indicators of computer hardware and
integrate the category discrimination information in multiple con-
software environment are as follows: CPU: Intel(R) Core(TM) i5-
volutional and pooling layers. The fully-connected layer is mathe-
6300HQ 2.3GHZ; RAM: 8G DDR4; Video Card: NVIDIA GeForce GTX
matically expressed as follows:
960M; Operating System: Windows 10 64bits; Development Envi-
J = f (r × a + u ) (6) ronment: Python 3.5; Tensorflow-gpu 1.10.0.
In this paper, recognition rate and average elapsed time are
In which, r is the input vector, a and u are the weight and used as the evaluation indexes. The recognition rate is defined as
bias of the fully-connected layer respectively, J is the output of the the ratio of the number of face images correctly recognized to the
fully-connected layer, and f( • ) is the activation function. total number of face images in the test set; the average elapsed
 J. Tang, Q. Su and B. Su et al. / Computer Methods and Programs in Biomedicine 197 (2020) 105622 5

Table 1
Convolutional Neural Network Structure.

Number of Layers Structure A Structure B Structure C Structure D Structure E

Type Kernel Size Type Kernel Size Type Kernel Size Type Kernel Size Type Kernel Size

1 C1 3 × 3 C1 3 × 3 C1 3 × 3 C1 3 × 3 C1 3 × 3
2 P1 2 × 2 P1 2 × 2 P1 2 × 2 P1 2 × 2 P1 2 × 2
3 C2 3 × 3 C2 3 × 3 C2 3 × 3 C2 3 × 3 C2 3 × 3
4 P2 2 × 2 P2 2 × 2 P2 2 × 2 P2 2 × 2 P2 2 × 2
5 I_C1 1 × 1 I_C1 1 × 1 I_C1 1 × 1 I_C1 1 × 1 I_C1 1 × 1
I_C2 3 × 3 I_C2 3 × 3 I_C2 3 × 3 I_C2 3 × 3 I_C2 3 × 3
I_C3 5 × 5 I_C3 5 × 5 I_C3 5 × 5 I_C3 5 × 5 I_C3 3 × 3
I_P1 3 × 3 I_P1 3 × 3 I_P1 3 × 3 I_P1 3 × 3 I_P1 3 × 3
6 P3 2 × 2 P3 2 × 2 P3 2 × 2 P3 2 × 2 P3 2 × 2
7 C3 1 × 3 C3 1 × 3 C3 1 × 3 P4 2 × 2 P4 2 × 2
8 C4 3 × 1 C4 3 × 1 C4 3 × 1 C3 1 × 3 C3 1 × 3
9 P4 2 × 2 P4 2 × 2 P4 2 × 2 C4 3 × 1 C4 3 × 1
10 FC – FC – C5 1 × 3 P5 2 × 2 P5 2 × 2
11 S – S – C6 3 × 1 FC – FC –
12 P5 2 × 2 S – S –
13 FC –
14 S –

time is defined as the average time to complete the recognition of

one face image.

7.2. Experimental results of face recognition

According to the above experimental dataset and the method

proposed in this paper, the experiment integrates 10 convolutional
neural networks in parallel with 5 different structures, in which
CNN1, CNN4, and CNN5 have the same structure (hereinafter re-
ferred to as Structure A), CNN2 and CNN3 have the same structure
(hereinafter referred to as Structure B), CNN6, CNN7, and CNN8
have the same structure (hereinafter referred to as Structure C),
the other two structures are for CNN9 (hereinafter referred to as
Structure D) and CNN10 (hereinafter referred to as Structural E)
respectively. Note that the combination of different CNN structures
is employed based on our testing results that compared the perfor-
Fig. 8. Schematic Diagram of Network Skip Connection.
mances of various combinations of CNN structures. For each com-
bination, we had executed around 30 times and recorded the aver-
aged accuracy. After several combinations are tested, the one with The recognition results of Yale-B dataset are demonstrated in
the best accuracy is chosen. The Inception in this method is made Fig. 11. The accuracy of single network is about 85%, and that of
up of three convolutional layers and one maximum pooling layer the integrated network is up to 97.51%, which is much higher than
in parallel. The detailed network structure is given in Table 1. C that of single convolutional neural network.
denotes the convolutional layer with a step size of 1; P denotes In summary, the parallel ensemble learning is beneficial to
the maximum pooling layer with a step size of 2; I_C denotes the strengthen the robustness of the algorithm and improve the final
convolutional layer in the Inception with a step size of 1; I_P rep- recognition rate.
resents the pooling layer in the Inception with a step size of 1;
FC denotes the fully connected layer, and S denotes the Softmax 7.3. Experimental results of pedestrian detection
logic regression function. Among them, Structure B, Structure D
and Structure E all contain a skip connection, as shown in Fig. 8. CBCL [12] and INRIA [13] pedestrian datasets are selected to test
Structure B jumps from the output of Max Pooling P3 to the input the effectiveness of our proposed method to improve the detection
of Max Pooling P4, while Structure D and Structure E jump from rate of pedestrian in this subsection. The CBCL data set contains
the output of Max Pooling P4 to the input of Max Pooling P5. 924 images, among which the size of each image is 64 by 128.
The recognition results of the iterations and accuracy of each Besides, the INRIA data set consists of 2416 images, where include
network in the ORL dataset are demonstrated in Fig. 9. Because 614 positives and 1218 negatives. Comparing to images in the CBCL
of the difference of CNN structure, the features learned are differ- data set, images in the INRIA are of higher resolution.
ent, and the accuracy of verification is also deviated. The data with First, we trained a CNN model to recognize pedestrian straight-
better learning effect in each structure are compared in Fig. 10. forwardly. After that, our proposed method is combined with the
Obviously, the accuracy of the integrated data is about 98%. Af- CNN model as a hybrid model, which aims at improving the pedes-
ter 2800 trainings, the accuracy of the test set is 100%, which is trian detection rate. As shown in Table 2, our hybrid model im-
much higher than that of single convolutional neural network. De- proves the detection rate of CNN from 96.3% to 99.5% on CBCL
spite the obvious change of expression and posture in the ORL data set, which claims the effectiveness of the Hybrid approach.
face dataset, the proposed method can still achieve high accuracy, In addition, as the INRIA data set contains images with higher res-
which proves that it is highly robust to expression and posture olution, the hybrid approach prefers such data set since high res-
change. olution carries more facial details to help the hybrid approach to
6 J. Tang, Q. Su and B. Su et al. / Computer Methods and Programs in Biomedicine 197 (2020) 105622

detect pedestrians, that leads to a 98.1% accuracy for the hybrid

approach on INRIA data set comparing to a 89.2% accuracy for the
CNN model.

7.4. Analysis and comparison of the results with other algorithms

In order to verify the performance, the proposed approach is

compared with PCA, CNN, HOG-CNN and LBP-CNN. PCA maps high-
dimensional face image to low-dimensional subspace, which makes
it easier to classify the face images. PCA is a popular face recogni-
tion method at present. In this paper, the PCA-based face recog-
nition is used as Contrast Scheme 1, hereinafter referred to as
Scheme 1. The face image recognition directly based on the CNN
is a common solution in face recognition. According to the ex-
perimental results in previous sections, if the convolutional neural
network does not use the ensemble learning scheme for classifi-
cation, the accuracy will be greatly reduced. Therefore, the CNN-
based parallel ensemble learning is used as Contrast Scheme 2,
hereinafter referred to as Scheme 2. The convolutional neural net-
work structure of Scheme 2 is consistent with that of Table 1, and
the difference between Scheme 2 and the proposed approach in
Fig. 9. Recognition Results of ORL Face Image Dataset.
the paper is that the texture features of the image input to CNN in
Scheme 2 are not extracted by LBP. The convolutional neural net-
works based on HOG features can also be employed for face recog-
nition. In the paper, the parallel ensemble learning of convolutional
neural network based on HOG features is used as Contrast Scheme
3, hereinafter referred to as Scheme 3. The convolutional neural
network structure of Scheme 3 is consistent with that of Table 1,
and the difference between Scheme 3 and the proposed method
in this paper is that Scheme 3 uses HOG to replace the LBP pro-
posed in this paper to extract the features. Note the 4th algorithm
LBP-CNN is similar to our proposed algorithm, yet the backbone of
CNNs in our algorithm is chosen from ResNet, which is able to go
deeper due to its skip connection comparing to the basic backbone
in LBP-CNN.
It can be seen from Table 3 that, in the ORL face image dataset
Fig. 10. Network Structure vs. Integrated Recognition Results. identification, the recognition rate of the proposed approach is sig-
nificantly higher than that of PCA (principal component analysis),
Table 2 HOG-CNN and LBP-CNN. At the same time, the recognition rate of
Experimental Results of Pedestrian Detection. the proposed method is a little higher than that of the traditional
Method Accuracy of CBCL/% Accuracy of INRIA/% convolutional neural network.
CNN 96.3 89.2
It can be seen in Table 4 that, in the identification of Yale-
Hybrid 99.5 98.1 B dataset, the accuracy of the proposed approach is 10% higher
than that of Scheme 2 ranked second, is over 50% higher than

Fig. 11. Recognition Results of Yale-B Face Database.

 J. Tang, Q. Su and B. Su et al. / Computer Methods and Programs in Biomedicine 197 (2020) 105622 7

Table 3 features and improve the face recognition accuracy. In CNN, the
Recognition Results of Each Method in ORL
Inception module is utilized to increase the width of the CNN,
Face Image Dataset.
the Batch Normalization is employed to reduce the training time,
Method Recognition Rate /% and the skip connection is employed to improve the accuracy of
Scheme 1 92.00 face recognition. The parallel ensemble learning makes the net-
Scheme 2 99.50 work structure no longer single, and greatly improves the accuracy
Scheme 3 93.50 and generalization ability of the proposed approach in the paper. In
Scheme 4 96.6
the experiments, we compared the proposed approach in this pa-
Proposed Method 100.00
per with other three methods, which are PCA, HOG-CNN and CNN
respectively. The final results illustrate that the proposed approach
Table 4
is more effective in face recognition, and its accuracy of face recog-
Recognition Results of Each Method in Yale-B
Face Image Dataset. nition is promising.

Method Recognition Rate /%

Declaration of Competing Interest
Scheme 1 38.23
Scheme 2 87.72 None.
Scheme 3 62.74
Scheme 4 96.7
Proposed Method 97.51 Acknowledgements

This work is supported by the Research and Development Fund

Table 5
The Average Elapsed Time of the approach Beijing Institute of Technology, Zhuhai (No. XK-2018-07), a grant
Proposed in the Paper. from Innovative Youth Foundation of Guangdong Educational Com-
mittee (No. 2018KQNCX346), and Guangdong government grant,
Database Average Elapsed Time /MS
(No. EF003/FST-FSJ/2019/GSTIC).
ORL 14
Yale-B 34 References

[1] Z. Tahira, H.M. Asif, Effect of averaging techniques on PCA algorithm and its
performance evaluation in face recognition applications, in: International Con-
that of PCA, is over 30% higher than that of HOG-CNN and is a ference on Computing, Electronic and Electrical Engineering (ICE Cube), 2018,
slightly higher than LBP-CNN. Because the illumination variation pp. 1–6.
of Yale-B face image dataset is relatively large, the recognition rate [2] D.N. Parmar, B.B. Mehta, Face recognition methods & applications, CoRR
abs/1403.0485 (2014).
will be reduced when the original image features are used. How-
[3] T. Ahonen, A. Hadid, M. Pietikainen, Face Description with Local Binary Pat-
ever, the proposed approach in the paper adopts LBP to extract terns: application to Face Recognition, IEEE Trans. Pattern Anal. Mach. Intell.
texture features, which greatly reduces the influence of illumina- 28 (12) (2006) 2037–2041.
[4] J. Liu, Y. Chen, S. Sun, Face recognition based on multi-direction local binary
tion changes. The robustness to the change of illumination of the
pattern, in: 3rd IEEE International Conference on Computer and Communica-
proposed method in this paper is stronger than that of the three tions (ICCC), 2017, pp. 1606–1610.
methods compared, and the extraction of face detail features and [5] M. Wang, Z. Wang, J. Li, Deep convolutional neural network applies to face
recognition rate of the proposed method are much better than that recognition in small and medium databases, in: 4th International Conference
on Systems and Informatics (ICSAI), 2017, pp. 1368–1372.
of Scheme 3. [6] H. Ahamed, I. Alam, M.M. Islam, HOG-CNN based real time face recognition,
In summary, the proposed method in this paper is superior in: International Conference on Advancement in Electrical and Electronic Engi-
to the three contrast schemes in both datasets. Compared with neering (ICAEEE), 2018, pp. 1–4.
[7] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition,
Schemes 1, 2 and 3, the proposed method in this paper has in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016,
stronger adaptability and higher accuracy in face recognition sit- pp. 770–778.
uations with large changes in illumination and expression. [8] T. Ojala, M. Pietikainen, T. Maenpaa, Multiresolution gray-scale and rotation
invariant texture classification with local binary patterns, IEEE Trans. Pattern
In the identification of ORL and Yale-B face image datasets, Anal. Mach. Intell. 24 (7) (2002) 971–987.
the average elapsed time of the proposed method in this paper is [9] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S.E. Reed, D. Anguelov, D. Erhan, V. Van-
shown in Table 5, and the recognition time is all less than 40 ms, houcke, A. Rabinovich., Going deeper with convolutions, in: IEEE Conference
on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1–9.
which is in accordance with the real-time processing requirements.
[10] S. Ioffe, C. Szegedy, Batch normalization accelerating deep network training
by reducing internal covariate shift, in: International Conference on Machine
8. Conclusion Learning (ICML), 2015, pp. 448–456.
[11] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception
architecture for computer vision, in: IEEE Conference on Computer Vision and
The face recognition approach based on the parallel ensem- Pattern Recognition (CVPR), 2016, pp. 2818–2826.
ble learning of LBP and CNN introduced in the paper adopts [12] CBCL pedestrian database, website: http://cbcl.mit.edu/software-datasets/
LBP to extract the texture features as training data for the par- PedestrianData.html. Last accessed: 10/27/2019.
[13] N. Dalal, B. Triggs, Histograms of oriented gradients for human detection, in:
allel CNN and finally is applicable to face recognition. LBP fea- IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2005,
tures are mainly utilized to extract facial texture features, be- pp. 886–893.
cause LBP can reduce the influence of illumination on facial