Real-Time Face Recognition Based On Pre-Identification and Multi-Scale Classification

ReView by River Valley This
Technologies
article has been accepted for publication in a future issue of this journal, but has not been fully IET Computer Vision
edited.
Content may change prior to final publication in an issue of the journal. To cite the paper please use the doi provided on the Digital Library page.
Real-time Face Recognition based on Pre-identification and Multi-scale

Classification
Weidong Min 1, Mengdan Fan 1, Jing Li 1*, Qing Han 1
1
School of Information Engineering, Nanchang University, Nanchang, China
*
corresponding. [email protected]
Abstract: In face recognition, searching a person’s face in the whole picture is generally too time-consuming to ensure high
detection accuracy. Objects similar to the human face or multi-view faces in low-resolution images may result in the failure
of face recognition. To alleviate the above problems, a real-time face recognition method based on pre-identification and
multi-scale classification is proposed in this paper. The face area is segmented based on the proportion of human faces in
the pedestrian area to reduce the search range, and faces can be robustly detected in complicated scenarios such as heads
moving frequently or with large angles. To accurately recognize small-scale faces, we propose the Multi-scale and Multi-
channel Shallow Convolution network (MMSCN) which combines a multi-scale mechanism on the feature map with a multi-
channel convolution network for real-time face recognition. It performs face matching only in the pre-identified face areas
instead of the whole image, therefore it is more efficient. Experimental results showed that the proposed real-time face
recognition method detects and recognizes faces correctly, and outperforms the existing methods in terms of effectiveness
and efficiency.
Key words: Face recognition, Real-time systems, Face pre-identification, Deep learning, Multi-scale, Multi-channel
Furthermore, to ensure high detection accuracy, further

1. Introduction filtering of the detected area is needed, which is time-
With the goal of matching the capabilities of human consuming.
vision [1], visual domain adaptation and generalization on For reliability, some researchers focus on improving
static images or video frames have received significant face detection methods, such as extracting hybrid features
attention in recent years. In particular, surveillance systems and improving classifiers [5]. However, when pedestrians
rely more and more on multi-view learning in the case that are far away from the cameras, the target domain mainly
the source domain is different from the target domain in consists of low-resolution images, while the source domain
visual recognition [2]. Nevertheless, this task is relatively mainly contains high-resolution images. Moreover, the head
difficult and it requires real time and reliability. moves more frequently than the body and recognizing multi-
Traditional face recognition methods recognize faces view faces becomes difficult when pedestrians look down,
by detecting human faces, tracking the faces, capturing a look up or look back. Other approaches improve face
frame of image, and then extracting features and matching matching methods by using multi-scale properties of the
faces in static images. For instance, Hamedani et al. [3] facial features and multi-modal information [6], but require
proposed a video-based face recognition method that first high-computational costs. In general, the existing methods
detected and tracked faces in videos, and then captured still encounter the problem in low-resolution face
video frames of a subject rotating his/her head. The recognition.
manifolds of video frames embedded in a high-dimensional Overall, there are mainly two difficulties in real-time
video space were extracted using neural network-based face recognition. One is that it is time-consuming to ensure
models. After that, the images containing rotating heads high detection accuracy; the other is the difficulty to
were composited using nonlinear manifold learning and recognize faces in low-resolution images. In view of the first
finally matched with the synthetic picture. On the other hand, difficulty, we propose a pre-identification scheme, which
some researches which integrate spatial and temporal combines pedestrian detection with face detection. To deal
probability models have been on the rise in recent years [4]. with the second difficulty, we propose a new deep learning
Tracking and identification in traditional face recognition network, the Multi-scale and Multi-channel Shallow
algorithms are implemented separately, while in the spatial Convolution network (MMSCN), for real-time face
and temporal based method, they are performed at the same recognition. The MMSCN combines face pre-identification
time, which is more real-time but a large amount of time with multi-scale feature maps and multi-channel shallow
information is needed both in the tracking and identification convolution network, which not only improves the detection
phases. In general, there are two main stages in the existing process of the traditional face recognition algorithms but
face recognition methods: i) detect human faces; ii) identify also optimizes the recognition algorithms.
the detected faces. To detect human faces, it is necessary to Firstly, in order to better capture small-scale target
search for the face in the whole image, and it may be domain pictures, we combine pedestrian detection with face
misjudged when other objects are similar to the face. detection, and track pedestrians as the person pre-
identification problem. The reason is that the basic principle
2018/10/24 18:22:16 IET Review Copy Only 2

Technologies
edited.
of machine learning for detecting an object is to traverse the Face recognition generally includes image capture,
whole picture with a search box to find the most similar face detection, face pre-processing, feature extraction, and
regions to the object. Large objects are more likely to be face classification. Luo et al. [5] proposed an adaptive skin
searched because there are less interfering pixels in the detection method using face location and facial structure
image [3]. Therefore, instead of searching faces directly, we estimation. The face location algorithm was developed to
obtain the area of the moving pedestrian by ACF features improve the reliability of face detection and extracted a face
and SVM classifier, and then estimate the face area based on region with a high proportion of skin. Common face
the proportion of human faces in the pedestrian area to recognition methods only use face detection to locate the
reduce the search range of the detector. Afterwards, we position of the face to search human faces. When
estimate the face area according to the center of each pedestrians are far from the cameras or in a complex
pedestrian area. With the moving of pedestrians, a human background, it is difficult to track moving faces from video
face region selector is used to correct candidate face regions. image frames. Several important enhancements made upon
This process is regarded as person pre-identification. the original framework related to the pre-processing, feature
Secondly, aimed at improving the face recognition calculation and training setup were described in [9]. On the
performance on relatively small face images, the feature other hand, pedestrian detection is widely used [10]. A
maps are down-sampled and central-clipped, respectively. pedestrian detection framework aided by information fusion
Then, these two kinds of processed feature maps are trained between binocular visions was proposed in [11], which has
separately by two network branches, and the features the potential of aggregating information from multiple
extracted from these two network branches are finally images to improve the detection on a single image. This
connected and used for classification. work describes several important enhancements made in the
The structure of this paper is as follows. Section 2 original framework related to the pre-processing steps,
introduces the related works. In Section 3, we propose a feature calculation and training setup. Overall, advanced
novel face recognition method. Section 4 shows the researches concerning face recognition are mostly based on
experimental results and numerical simulations. Finally, machine learning, which finds the objects by traversing the
conclusions and future work are given in Section 5. whole image with a search box. Thus, large objects are more
likely to be searched because there are less interfering pixels
2. Related Works in the image [3].
Originally, video-based face recognition algorithms Recently, deep learning-based algorithms have been
identify human faces by the following steps: i) face widely applied in computer vision research area [3]. Wang
detection; ii) face tracking; iii) feature extraction; and iv) et al. [12] proposed a joint method of priori convolutional
face matching. Ren et al. [4] proposed a framework for neural networks at the superpixel level (called as “priori s-
dynamic scheduling for energy minimization (DSE) that CNNs”) and soft restricted context transfer. It is worth
leverages this emerging hardware heterogeneity. For noting that a priori s-CNNs model that learns priori location
instance, Schroff et al. [6] presented a system, called information at the superpixel level was proposed to describe
FaceNet, which directly learns a mapping from face images various objects discriminatingly. A network trained end to
to a compact Euclidean space where distance directly end for better optimization which can be used for
corresponds to a measure of face similarity. To assess the crowdedness regression tasks including congestion-level
performance of DSE, the authors built a face detection detection and crowd counting was proposed in [13]. Wang
application based on the Viola-Jones classifier chain and et al. [14] proposed a siamesed fully convolutional network
conducted experimental studies via heterogeneous processor (named as “s-FCN-loc”) based on the VGG-net architecture,
system emulation. On the other hand, several methods based which is able to consider the RGB-channel, semantic
on the integration of spatial and temporal probability models contour and location prior simultaneously to segment the
have been on the rise in recent years. For instance, Retter et road region elaborately. Although the deep learning methods
al. [7] presented the human observers with natural images of mentioned above have achieved great success in those
objects at a fast periodic rate of 12.5 Hz, i.e., every 80 ms, computer vision tasks, the network structures are complex to
uncovering that the neural spatio-temporal dynamics of a certain extent, and not all deep learning networks are
category-selectivity in a rapid stream of natural images went suitable for face recognition problems. It is worth thinking
well beyond previous evidence obtained from spatially and how to make the network structure simpler.
temporally isolated stimuli. Demirkus et al. [8] developed a In order to improve the face recognition rate, multi-
fully automatic hierarchical and probabilistic framework view face detection [13] has been widely proposed in face
that models the collective set of frame class distributions recognition. In the video-based face recognition process,
and spatial feature information over a video sequence. It is with the uncertainty of pedestrian movement in the target
flexible enough to be applied to any facial classification area, the face is likely to appear with different sizes, so it is
tasks. Tracking and identification in the former method are necessary to normalize the size of face images. But if the
performed separately, while in the latter they are performed size of the captured human face from a video is relatively
at meanwhile. Therefore, the latter method is more efficient small, the obtained face image needs to be enlarged, which
but a large amount of time information is needed both in the reduces image resolution. To solve this problem, a bottom-
tracking and identification phases. Overall, the research of up saliency detection method that unites the syncretic merits
face recognition so far mainly focuses on the optimization of of sparse representation and multi-hierarchical layers was
face recognition algorithms on static images, and the proposed. In contrast to most pre-existing sparse-based
optimization of the spatial and temporal probability models approaches that only highlight the boundaries of a target, the
of video-based face recognition. method highlights the entire object even if it is large [15].
He et al. [16] equipped the networks with spatial pyramid
2

Technologies
edited.
pooling. The network structure, called SPP-net, can generate 3. Proposed Method
a fixed-length representation regardless of image size/scale. The key steps of the real-time video-based face
Pyramid pooling is also robust to object deformations. recognition are as follows: 1) face detection: detect and
Although the recognition performance is improved, the mark the faces in video frames; 2) face tracking: regard the
detection algorithm in the above methods remains the same. coordinates of detected human faces as the initial state of the
Searching a person’s face in the whole picture is time- tracking target, and then track the faces; 3) feature
consuming which requires complex calculation and could extraction: extract the face features such as principal
bring in the delay of the algorithm. In contrast, a components from the entire face; 4) feature dimension
computationally efficient face recognition method based on reduction: reduce the extracted high-dimensional features to
multi-scale images could be a good substitute. Overall, it is low-dimensional ones; 5) face recognition: perform face
not easy to recognize low-resolution faces for the existing classification based on face features.
face recognition methods. On the basis of the above contents, we propose a
To this end, this paper aims to alleviate the issues real-time face recognition method based on pre-
discussed above, which are low face detection and identification detection and multi-scale classification.
recognition rate, and high time consumption. The
contributions of this paper are as follows: 3.1. Proposed Pre-identification detection
1) We perform face detection based on pre-
identification, in which the face area is captured based on 3.1.1. Overall process: The flowchart of the whole
the proportion of human faces in the pedestrian area. It algorithm is depicted in Fig. 1. Firstly, the ACF features are
could reduce the search range, and the faces can be robustly extracted and classified by SVM. Secondly, NMS is used to
detected in complicated scenarios such as heads moving improve the algorithm by getting multiple overlapped
frequently or with large angles. borders (bounding box) and reducing them to only one
2) We propose a novel deep learning network border. Thirdly, the face area is estimated based on the
MMSCN to recognize faces in multiple views by combining proportion of the pedestrian area to reduce the search range,
multi-scale feature maps with a multi-channel shallow which is regarded as pre-identification.
convolution network.
Fig. 1. Detection based on pre-identification.

sequences. In the real-time application scenarios, the prior
3.1.2. Principle of pre-identification detection: information of the scene can be used to determine the
According to previous sections, general face recognition potential area of the face, which reduces the search range of
methods search human faces throughout the whole image. the detector. Considering this, we obtain the moving
When pedestrians are far from the cameras or in a complex pedestrian area by a pedestrian detection algorithm and
background, it is hard to track moving faces from video estimate the face area based on the proportion of the
3

Technologies
edited.
pedestrian area to reduce the search range, which is referred getting multiple overlapped borders (bounding box) and
as pre-identification. reducing them to only one border.
Pedestrian detection involves the method based on ACF is one of the most advanced feature extraction
background modelling and the method based on classifiers. algorithms [29]. The ACF features are robust by
Background modelling can be divided into background superimposing the LUV color, the gradient amplitude and
subtraction and frame differencing. Background subtraction the gradient direction features. SVM can be applied to
detects moving objects by comparing the current frame with solving other problems, but in this section it is mainly used
the background reference model through the determination based on the core idea introduced in [30]. The NMS method
of the change of gray level [17-19], or use the histogram and is mainly used to eliminate redundant frames through
other statistical information to segment the moving object searching for local maxima and suppress non maxima
[20-22]. It is one of the most commonly-used methods of elements through getting rid of the redundant box based on
moving-object detection in video sequences. Frame the classifier score for each candidate [31].
differencing [23, 24] is a method to obtain the target contour
through differential operation of adjacent video frames. For
fast-moving objects, smaller intervals need to be selected; 3.1.3 Pre-identification mechanism: The human face area
otherwise the objects in two adjacent frames without overlap is estimated according to the proportion of the pedestrian
will be detected as two separate objects. For slow-moving area to reduce the search range, which is regarded as pre-
objects, the larger time difference should be chosen; identification. With the moving of the pedestrians, a face-
otherwise the objects in two adjacent frames that completely region selector is used to correct face candidate regions,
overlap will not be detected. which is described as follows. The whole process of face
The pedestrian detection methods based on pre-identification before face detection in each image frame
classifiers are divided into AdaBoost, SVM and deep is shown in Fig. 1. Our face detection algorithm based on
learning methods. For instances, the cascade implementation pre-identification is described totally in Algorithm 1.
of the additive KSVM (AKSVM) was proposed in [25].
AKSVM avoids kernel expansion by using lookup tables,
and it is implemented in a cascade form, thereby speeding Algorithm 1 Face detection
up pedestrian detection. The cascade implementation is
trained by a genetic algorithm such that the computation
Input: Image or video
time is minimized, whereas the detection accuracy is 1: Use Depth-2 decision tree as the weak classifier and
maximized. Wu et al. [26] proposed a semi-supervised soft-cascade structure to extract feature z .
approach for training deep convolutional networks on 2: The training samples are located on both sides of
partially labelled data. It is hypothesized that the the hyperplane.
components of the auxiliary detector capture essential
3: if ( z is linear separable)
human characteristics that are useful for constructing a
scene-adapted detector. Dominguez-Sanchez et al. [27] 4: A total T = (x1 , y1 ),(x2 , y 2 ),(x3 , y3 )...
proposed a CNN-based technique that leverages current samples, the hyperplane is w  x + b = 0 , the
pedestrian detection techniques, which are used as an input geometric distance from the sample point to the
for the proposed modified versions of various state-of-the-
art CNN networks, such as AlexNet, GoogleNet, and hyperplane is: i =
(
y  y ( x i ) y i  wT   ( x i ) + b);
ResNet. The detection methods based on deep learning are w w
better in recognition performance. In summary, the detection 5: else
efficiency of the derivatives of these methods is basically
the same [28], but SVM is more concise than the AdaBoost, 6: Look for a point on the horizontal axis and
and the number of parameters is less than that of CNN. calculate the function value of this point. It is
According to the methods mentioned above, we a time curve we know, and its function
detect the moving pedestrian area by ACF features [29] and expression can be written as:
SVM classifier [30], and then use the non-max suppression g (x) = c0 + c1 x + c2 x 2 .
(NMS) [31] to remove redundant frames. Finally, the face 7: Let vector y and a be as the follows:
area based on the proportion of the pedestrian area is
estimated to reduce the search range, which is regarded as a = a1 ,a2 ,a3 T = c0 ,c1 ,c2 T
pre-identification. The entire algorithm will be described in
Section 3.1.4. In order to further verify the effectiveness of 
y = y1 , y 2 , y 3 T = 1, x, x 2 
T
then we have:
the pre-identification scheme, we compare it with the g (x ) = ay
.
detection process in advanced face detection algorithms in
terms of accuracy and time efficiency in Section 4. 8: Left-most candidate boxes of the input sequence
stand for the candidate boxes.
3.1.2 Pedestrian detection: Based on the comprehensive 9: if ( I  j   I  j + 1 and I  j   I  j − 1 )
MaximumAt (i ) ;
comparison of Section 3.1.1, we combine ACF and SVM to
10:
detect moving objects. Firstly, the ACF features are
extracted and classified by SVM. Secondly, non-max 11: else i is not an local maximum
suppression (NMS) is used to improve the algorithm by

Technologies
edited.
12: Divide the whole image into p  q regions, where Network: Fig. 2 shows the MMSCN. In order to better
each region corresponds to a selector θ : Where H is recognize the face at different scales, inspired by [32], we
combine down sampling and central cutting. Down
the height of the pedestrian and h is the height of the sampling can reduce the resolution of the feature map and
human face; prepare for recognizing small-scale images with low
13: Get the upper-left corner of the pedestrian area resolution. The resolution of the original feature map is
and the lower-right corner through the target maintained after central cutting, but the scale has also been
detection and tracking algorithms; declined. To reduce the computational complexity, the patch
14: Calculate the center of each target in the region α ij ; is cut out from the feature map instead of the original image.
15: Estimate human face regions by the following two At the same time, the network structure is made into a dual
formulas: The upper-left corner of the area which may channel mode in order to simplify the parameter setting.
contain human faces is: (x1 + 1, y1 + 1) ; The lower-right
Multi-scale processing: Here, the feature maps are processed
corner of the area which may contain human faces is:
( )
in multiple scales. Firstly, the original image is used to
(x2 − 1), y1 + θij  ( y2 − y1 − 1) search out the candidate window of face. Secondly, the
entire image to be identified is entered to CNN, feature
3.2. Proposed Multi-scale classification maps are obtained, and then pooled by different methods,
i.e., down sampling and central cutting. The idea of multi-
Before recognizing the face, we pre-process face scale processing is to split the original feature map of 64*64
images to improve the image quality by eliminating the into two 32*32 sub-maps. The first sub-map is directly
irrelevant information in the image so as to enhance the obtained through down sampling of the original feature map.
detection ability. Pre-process includes geometric The second sub-map is obtained by using central cutting to
transformation (rotate and translate the image) and cut out a 32*32 sub-map in the center of the origin feature
normalization (clip the image), histogram equalization map, which is shown as the light blue area in Fig. 2.
(improve brightness and contrast), smoothing (reduce noise). Source domain and Target domain: D = x , P( X ) , D
After that, we propose a new deep learning network represents the domain, x represents the feature space, and
MMSCN for real-time face recognition which combines X = x1 , x2 ,...xn  x . P(x ) represents the marginal
multi-scale feature maps and multi-channel shallow
convolution network. probability distribution, and xi is the binary feature of the i
Different from static images, it is not easy to -th pixel block .
determine how pedestrians move in the target area in video
sequences. Since face images may have substantial changes, Learning tasks: A task T can be made up of a tag space Y
normalization should be considered. Here, images of small and an objective function f T () . The objective function can
sizes are enlarged, and the resolution is reduced which could be expressed as a form of conditional probability
lead to wrong recognition results. In order to solve the above distribution P(Y x ) . For two categories, it is either true or
problems, we propose the MMSCN, as shown in Fig. 2. It
consists of two network branches. The feature maps are false.
down-sampled and central-clipped, respectively. Then, these
two kinds of processed feature maps are trained separately Learning goals: The source domain Ds is integrated with
by two network branches, and the features extracted from the features of different-scale images. The target domain Dt
these two network branches are finally connected and used mainly contains small-scale pictures. The purpose is to
for classification. improve the classification result of the objective function
f T () in D t .
4. Experimental and Numerical Simulations

The experimental environment used in the
experiments is: Intel Celeron i5-2400 @3.10GHz CPU, 4GB
internal storage, Windows 7 32bit operating system, Axis
215 PTZ network camera, 30 frames per second as the frame
rate for network cameras.
Since our algorithm is based on pedestrian
detection, to demonstrate the superiority of our algorithm,
the training samples of the database must contain both the
whole body and the face of the pedestrians. Due to the lack
of such a dataset, we collected a dataset containing 5
different subjects. For each subject, 900 frames are sampled.
Besides, we made a standard dataset by fusing the data from
the processed LFW dataset [33] and the processed PIE
dataset [33] together since they are similar in type to some
Fig. 2. Our proposed MMSCN extent. In the processed LFW dataset, there are more than
13,000 pictures in total, containing 5,765 different identities.
5

Technologies
edited.
The PIE dataset contains 750,000 face images from 337

different identities with different attitudes, expressions and
illumination changes. In order to better reflect the
superiority of the proposed algorithm on multi-scale face
recognition, the images in the test set of the standard dataset
are sampled to 28  28 . Then, the images at this scale will
be normalized to the input size before being tested by the
model. Padding after down-sampling greatly reduces the
resolution of the original map, and thus can be used to verify
whether low-resolution data can be accurately detected.
4.1. Face detection efficiency with pre-

identification
With the moving of pedestrians, the detection

method uses human face region selector to correct face
candidate regions, as described in section 3.1. The
correction operator obtains faces regions through the Fig. 4. Qualitative analysis
segmented regions, and dynamically adjusts the detection
window in real-time to ensure the detection effect. Fig. 3 In Fig. 4, it can be seen that both MTCNN and
shows the candidate face regions obtained by the correction Viola-Jones cannot filter out the background when the face
operator at different moments on the self-collected dataset. is fairly small, which may lead to misjudge. On the contrary,
As we can see, no matter what it is, e.g., a side face, far- the algorithm proposed in this paper can solve this problem
distance face, near-distance face or face with partial-body very well. The results of the quantitative analysis are shown
unseen, the pre-identification method could detect the face in Fig. 5 and Table 1. Because of the lack of a standard face
accurately. detection dataset that contains the whole body, we use the
ROC curve and fps as criteria to evaluate the algorithms
based on the self-collected dataset (Video frame rate is 30
fps). Fig. 5 shows the ROC curves of the proposed detection
algorithm and the advanced detection algorithms on the self-
collected dataset. It can be seen that pre-identification
achieves better performance.
0.8
0.6
TPR
0.4
0.2 Viola-Jones(0.75)
MTCNN(0.83)
Pre-identification(0.89)
0
0 0.2 0.4 0.6 0.8 1
Fig. 3. Candidate face regions obtained in different FPR
moments
Fig. 5. ROC curves of the proposed detection algorithm
To demonstrate the superiority of pre-identification, and the advanced detection algorithms on the self-collected
we compare the detection results with two of the most dataset.
advanced face detection algorithms, which are MTCNN [6]
and Viola-Jones [4] by qualitative and quantitative analysis.
In particular, because the standard dataset does not contain Table 1 Comparison of time consumption
the whole body of pedestrians, we implement qualitative
analysis only on our newly collected dataset. Face detection algorithm Frame rate (fps)
V-J 27.509
MTCNN 26.748
Pre-identification scheme 29.779

Technologies
edited.
4.2. Recognition effect of MMSCN state-of-the-arts face recognition methods. They are
DeepFace [34], FaceNet [6], DeepID [35], and L-Softmax
In order to verify the effectiveness of the MMSCN, [36]. The results are shown in Fig. 7. It can be seen that the
we conduct qualitative and quantitative comparisons with proposed method can successfully and accurately detect the
some of the state-of-the-arts face recognition methods on face once it just appeared in the video. As seen from Fig. 7,
processed LFW dataset [33] and processed PIE dataset [33]. although the FaceNet algorithm has obtained the highest
The training set accounted for seventy percent, and the recognition accuracy so far [6], its detection method is not
testing set was thirty percent. In order to better reflect the robust enough. Moreover, its structure is relatively complex,
superiority of the proposed algorithm on multi-scale face which requires a large amount of computation. In summary,
recognition, the images in the testing set of the standard the algorithm proposed in this paper can accurately detect
dataset are sampled to 28  28 . Moreover, the image at this faces, and it obtains a higher recognition rate compared with
scale will be normalized to the input size before being tested advanced methods. Moreover, the model is simple and
in the models. This greatly reduces the resolution of the suitable for real-time face recognition systems.
original map. Fig. 6 shows the comparison results on the
original testing set and the small-scale testing set.
Fig. 7. Qualitative results of recognition
5. Conclusion
A real-time face recognition method based on pre-
identification and multi-scale classification is proposed in
this paper. Firstly, we combine pedestrian detection with
Fig. 6. Average ROC curves of recognition algorithms on face detection, and track pedestrians as person pre-
the standard dataset. identification. After obtaining the area of the moving
pedestrian by ACF features and SVM, we estimate the face
As we can see from Fig. 6, although FaceNet area based on the proportion of the pedestrian area to reduce
obtains better classification performance on the original the search range of the detector. Secondly, aimed at
testing set, the MMSCN performs better on the small-scale improving the face recognition performance on relatively
testing set. The greater the area of the area under curve small face images, we propose Multi-scale and Multi-
(AUC) is, the better the performance of the algorithm. Fig. 6 channel Shallow Convolution network (MMSCN) to
proves that our algorithm has higher recognition rate on recognize the face. Experimental results showed that the
small-scale images. proposed face recognition method outperforms the existing
methods in terms of effectiveness and efficiency.
4.3. Overall recognition effect There still exists several open studies in the above
In order to test the robustness of the whole research. In regarding of the pre-identification method, the
algorithm, we compare the recognition results of the four potential studies will be oriented to studying better
7

Technologies
edited.
mechanisms for predicting the range of faces. Furthermore, [10] Mazzon, R., Tahir, S., Cavallaro A.: 'Person re-
the depth of the MMSCN structure could be increased. We identification in crowd', Pattern recognition let., 2012, 33,
have only made a preliminary study with limited (14), pp. 1828 – 1837.
experiments that prove the effectiveness of the strategy.
[11] Ren H., Li, Z.: 'Object detection using boosted local
6. Acknowledgments binaries', Pattern Recognition., 2016, 60, (1), pp. 793 – 801.
This work is supported by National Natural Science
Foundation of China under Grant 61463032, 61703198, [12] Wang, Q., Gao, J., Yuan, Y.: 'A Joint Convolutional
61762061 and the Natural Science Foundation of Jiangxi Neural Networks and Context Transfer for Street Scenes
Province, China under Grant 20161ACB20004. Labeling', IEEE Trans. Intell. Transp. Syst., 2018, 19, (5),
pp. 1457 – 1470.
7. References [13] Wang, Q., Wan, J., Yuan, Y.: 'Deep Metric Learning
for Crowdedness Regression', IEEE Trans. Circ Syst. Video
[1] Bondi, L., Baroffio, L., Cesana, M., Tagliasacchi, M., T., 2017, pp.
Chiachia, G., Rocha, A.: 'Rate-energy-accuracy optimization
of convolutional architectures for face recognition', J vis [14] Gao, J., Wang, Q., Yuan, Y.: 'Embedding structured
commun image represent, 2016, 36, (1), pp. 142–148. contour and location prior in siamesed fully convolutional
networks for road detection' Proc. Int. Conf. Robotics and
[2] De-la-Torre, M., Granger, E., Sabourin, R., Gorodnichy, Automation, Singapore, Singapore, May 2017, pp. 219–224.
D.O.: 'Adaptive skew-sensitive ensembles for face
recognition in video surveillance', Patt Recog, 2015, 48, (11), [15] Hu, Z., Zhang, Z., Sun, Z., Zhao S.: 'Salient object
pp. 3385–3406. detection via sparse representation and multi-layer contour
zooming', Iet Computer Vision., 2017, 11, (4), pp. 309 –
[3] Hamedani, K., Seyyedsalehi, S. A., Ahamdi R.: 'Video- 318.
based face recognition and image synthesis from rotating
head frames using nonlinear manifold learning by neural [16] He, K., Zhang, X., Ren, S., Sun. J.: 'Spatial Pyramid
networks', Neural Comput and Applic. 2015, 27, (6), pp. 1– Pooling in Deep Convolutional Networks for Visual
9. Recognition ', IEEE Trans. Pattern Anal. Mach. Intell., 2014,
37, (9), pp. 1904 –1916.
[4] Ren, S., Deligiannis, N., Andreopoulos, Y., Islam, M.,
Schaar, M.: 'Dynamic Scheduling for Energy Minimization [17] Sato, Y. D., Nagatomi, T., Horio, K., H.: 'Miyamoto,
in Delay-Sensitive Stream Mining', IEEE Trans Signal The Cognitive Mechanisms of Multi-scale perception for the
Process., 2014, 62, (20), pp. 5439–5448. recognition of extremely similar faces', Cogn. Comput.,
2015, 7, (5), pp. 501 – 508.
[5] Luo, Y., Guan, Y. P.: 'Adaptive skin detection using face
location and facial structure estimation', Iet Computer [18] Liu, X., Zhao, G., Yao, J., Qi S, 'Background
Vision., 2017, 11, (7), pp. 550–559. subtraction based on low-rank and structured sparse
decomposition', IEEE Trans. Image. Processing., 2015, 24,
[6] Schroff, F., Kalenichenko, D., Philbin, J.: 'FaceNet: A (8), pp. 2502 – 2514.
unified embedding for face recognition and clustering', Proc.
Int. Conf. Computer Vision and Pattern Recognition [19] Xie, Y., Yang, L., Sun, X., Wua, D., Chen, Q, Yong,
(CVPR), MA, USA, 2015, pp. 815– 823. M., 'An auto-adaptive background subtraction method for
Raman spectra', Spectrochimica Acta Part A: Molecular and
[7] Retter, T. L., Rossion, B.: 'Uncovering the neural Biomolecular Spectroscopy., 2016, 161, (1), pp.58 – 63.
magnitude and spatio-temporal dynamics of natural image
categorization in a fast visual stream', Neuropsychologia., [20] Ge, W., Guo, Z., Dong, Y., Chen Y, 'Dynamic
2016, 91, (1), pp. 9–28. background estimation and complementary learning for
pixel-wise foreground/background segmentation', Pattern
[8] Demirkus, M., Precup, D., Clark J. J., Member S., Arbel Recognition., 2016, 59, (1), pp. 112 – 125.
T.: 'Hierarchical Spatio-Temporal Probabilistic Graphical
Model with Multiple Feature Fusion for Binary Facial [21] Huang, M., Chen, Y., Ji, W., Miao C., 'Accurate and
Attribute Classification in Real-World Face Videos', IEEE Robust Moving-Object Segmentation for Telepresence
Trans. Pattern Anal. Mach. Intell., 2016, 38, (6), pp. 1185 – Systems', ACM Trans. Intell. Syst. Tech., 2015, 6, (2), pp. 1
1203. – 17.
[9] Bilal M.: 'Algorithmic optimisation of histogram [22] Liang, C. W., Juang, C. F., 'Moving Object
intersection kernel support vector machine-based pedestrian Classification Using a Combination of Static Appearance
detection using low complexity features', Iet Computer Features and Spatial and Temporal Entropy Values of
Vision., 2017, 11, (5), pp. 350 – 357. Optical Flows', IEEE Trans. Intell. Transpsyst., 2015, 16,
(6) , pp. 3453 – 3464.

Technologies
edited.
[23] Zhang, Y., Lu, H., Zhang, L., Ruan, X., 'Combining [36] Liu W, Wen Y, Yu Z, and Yang M., 'Large-margin
motion and appearance cues for anomaly detection', Pattern softmax loss for convolutional neural networks', Proc. Int.
Recognition., 2016, 51, (1), pp. 443 – 452. Conf. Machine Learning (ICML), 2016, pp. 507-516.
[24] Shi, X., Shan, Z., Zhao, N., 'Learning for an aesthetic Author names and affiliations
model for estimating the traffic state in the traffic video',
Neurocomputing., 2016, 18, (1) , pp. 29 – 37. Weidong Min
[25] Baek, J., Kim, J., Kim, E., 'Fast and Efficient Pedestrian He obtained his BE, ME and PhD of
Detection via the Cascade Implementation of an Additive computer application at Tsinghua
Kernel Support Vector Machine', IEEE Trans. Intell. University in China in 1989, 1991 and
Transpsyst., 2017, 18, (4), pp. 902 – 916. 1995, respectively, on the research
subjects of computer graphics, image
[26] Wu, S., Wang, S., Laganiere, R., Liu, C., Wong, H. S., processing and computer aided geometric
Xu, Y., 'Exploiting Target Data to Learn Deep design. He was an assistant professor of Tsinghua
Convolutional Networks for Scene-Adapted Human University from 1994 to 1995. From 1995 to 1997 he was a
Detection', IEEE Trans. Image. Processing., 2018, 27, (3), postdoctoral researcher at University of Alberta, Canada.
pp. 1418 – 1432. From 1998 to 2014 he worked as a senior researcher and
senior project manager at Corel and other companies in
[27] Dominguez-Sanchez, A., Cazorla, M., 'Orts-Escolano Canada. In recent years, he cooperated with School of
S. Pedestrian Movement Direction Recognition Using Computer Science & Software Engineering, Tianjin
Convolutional Neural Networks', IEEE Trans. Intell. Transp. Polytechnic University, China. From 2015 he is a professor
Syst., 2017, 18, (12), pp. 3540 – 3548. at School of Information Engineering, Nanchang University,
China. He is a Member of “The Recruitment Program of
[28] Htike, K. K., Hogg, D., 'Adapting pedestrian detectors Global Expert” of Chinese government. He is an executive
to new domains: A comprehensive review', Eng. Applartif. director of China Society of Image and Graphics. His
intell., 2016, 50, (1), pp. 142–158. current research interests include computer graphics, image
and video processing, distributed system, software
[29] Yang, B., Yan, J., Lei, Z., Li, S. Z.: 'Aggregate channel engineering and network management.
features for multi-view face detection', Proc. Int. Conf.
Biometrics (IGCB), Clearwater, FL, USA 2014, pp. 1– 8.
[30] Wu, J., Yang, H.: 'Linear Regression-Based Efficient Mengdan Fan
SVM Learning for Large-Scale Classification', IEEE Trans.
Neural. Netw. Learn. Syst. 2017, 26, (10), pp. 2357 – 2369. She obtained her BE of computer
application at Jiangxi Agricultural
[31] Song, Y., Glasbey, C. A., Polder, G.: 'Non-destructive University in China in 2015. She is a
automatic leaf area measurements by combining stereo and postgraduate at Nanchang University in
time-of-flight images', Iet Computer Vision., 2014, 8, (5), pp. China now, on the research subject of abnormal behavior
391 – 403. detection in video surveillance.
[32] Zagoruyko, S., Komodakis, N., 'Learning to Compare

Image Patches via Convolutional Neural Networks', in Int.
Conf. Computer Vision and Pattern Recognition (CVPR). Jing Li
MA, USA, 2015, pp. 4353 – 4361.
Jing Li obtained her PhD degree in
[33] Soltanpour, S., Boufama, B., Wu, Q., 'A Survey of Electronic and Electrical Engineering from
Local Feature Methods for 3D Face Recognition', Pattern the University of Sheffield, UK, in 2012.
Recognition, 2017, pp. 72–87. Before joining Nanchang University as an
Associate Professor, she was a Research
[34] Taigman Y, Yang M, Ranzato M, Wolf L., 'DeepFace: Associate at the University of Sheffield.
Closing the Gap to Human-Level Performance in Face Her research interests include content-based image retrieval,
Verification', Proc. Int. Conf. Computer Vision and Pattern object recognition, visual tracking and scene understanding
Recognition (CVPR), Columbus, Ohio, June 2014, pp. in complex environments. She has authored or co-authored
1701-1708. in various journals, such as IEEE Transactions on Industrial
Informatics, Information Sciences (Elsevier), etc.
[35] Sun Y, Liang D, Wang X, Tang X., 'DeepID3: Face
Recognition with Very Deep Neural Networks', Proc. Int. Qing Han
Conf. Computer Vision and Pattern Recognition (CVPR),
MA, USA, 2015, pp. 1-5.
She obtained her BE and ME of computer
application at Tianjin Polytechnic
University in China in 1997 and 2006,
9

Technologies
edited.
respectively. She is now an associate professor at School of

Information Engineering, Nanchang University, China. Her
research interests include image and video processing,
network management.
10

Real-Time Face Recognition Based On Pre-Identification and Multi-Scale Classification

Uploaded by

Copyright:

Available Formats

Real-Time Face Recognition Based On Pre-Identification and Multi-Scale Classification

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Real-Time Face Recognition Based On Pre-Identification and Multi-Scale Classification

Uploaded by

Copyright:

Available Formats

ReView by River Valley This

Real-time Face Recognition based on Pre-identification and Multi-scale

Furthermore, to ensure high detection accuracy, further

2018/10/24 18:22:16 IET Review Copy Only 2

2018/10/24 18:22:16 IET Review Copy Only 3

Fig. 1. Detection based on pre-identification.

2018/10/24 18:22:16 IET Review Copy Only 4

2018/10/24 18:22:16 IET Review Copy Only 5

4. Experimental and Numerical Simulations

2018/10/24 18:22:16 IET Review Copy Only 6

The PIE dataset contains 750,000 face images from 337

4.1. Face detection efficiency with pre-

With the moving of pedestrians, the detection

2018/10/24 18:22:16 IET Review Copy Only 7

Fig. 7. Qualitative results of recognition

2018/10/24 18:22:16 IET Review Copy Only 8

2018/10/24 18:22:16 IET Review Copy Only 9

[32] Zagoruyko, S., Komodakis, N., 'Learning to Compare

2018/10/24 18:22:16 IET Review Copy Only 10

respectively. She is now an associate professor at School of

2018/10/24 18:22:16 IET Review Copy Only 11

You might also like