2020 16th International Conference on Computational Intelligence and Security (CIS)

A Mobile Application of Face Recognition Based on Android Platform

Wei Deng, XinLun Zhang*, Zhe Jiang

School of Computer Science and Engineering
GuiLin University of Aerospace Technology
2020 16th International Conference on Computational Intelligence and Security (CIS) | 978-1-6654-0445-7/20/$31.00 ©2020 IEEE | DOI: 10.1109/CIS52066.2020.00068

GuiLin, China
*e-mail: [email protected]

Abstract—With the popularization of smart phones, a large mobile phone’s camera to take a face photo, then enter their
number of mobile applications (Apps) have been developed personal information to complete the registration. After
and become more and more important in people's lives. registration users could log in the App with their own photo.
However, most of them need to verify user identify by their We implement three face recognition algorithms in our
username and password to unlock its full functions, which is an App. In addition, in order to find out which one is more
inconvenient operation for users. In order to solve this problem, suitable for mobile Apps, we collect a data set of human
this paper develops an App which employs face recognition faces, and use it as a benchmark in our experiment to choose
technology to check one’s identify. Experiments are conducted the most robust algorithm. The experiment results shows that
and experimental results demonstrate that our app has good our App has a reasonable recognition rate and high
performance in terms of recognition rate and time efficiency.
efficiency, thus it is suitable for practical use.
Meanwhile, we collect a face dataset for checking one’s identify.
II. APPLICATION FRAMEWORK
Keywords- mobile apps; face recognition; face dataset;
Considering the software and hardware limitations of
I. INTRODUCTION mobile phones, we adopt a C/S architecture [6], in which the
In recent years the smart phones have now become mobile phone (the client) interacts with users, collects human
necessities in people’s daily life. Especially after the launch faces, and normalizes the collected images, while the server
of the Android system and the Apple IOS SDKs, various side is responsible for face feature extraction, face image
Apps on smart phones began to grow rapidly [1]. Among storage and other functions.
them, some Apps related to E-commerce such as mobile The functional structure of our App is shown in Figure 1.
payment and mobile banking applications are particularly The left is the mobile phone client developed by Android
critical because they involve the security of users’ property. SDK, and the right is the Web server built by JavaEE. The
To protect users’ personal information, usually these Apps face recognition function has two main modules. One is the
ask users to manually type in their username and password to off-line facial feature extraction module, which extracts the
authenticate their identity. However, there are some certain image features of collected face data, stores them in the face
drawbacks in this traditional authentication method. For image feature library as shown in Figure 1, and then builds a
instance, it makes users feel confused that they need to linear index of these features. The other one is the online
remember different accounts and passwords to login registration and login module, in which the mobile App
different Apps, and there will be potential security problems client collects user face images and personal information,
if a user sets the same username and password for all Apps. encapsulates the HTTP packets, and then submits them via
Therefore, it is an urgent issue for mobile Apps to develop a HttpClient [7] (an open source development kits of Apache);
new mode of user authentication. The server side is responsible for receiving the messages,
Due to the development of biotechnology [2], some extracts face features, and matches the corresponding face in
biometric methods have been widely used in people's daily the existing linear index structure; After that, the matching
lives, including iris recognition [3], fingerprint recognition [4] result is encapsulated into the JSON format, and then sent
and face recognition [5]. Iris recognition and fingerprint back to the mobile client to complete the entire process.
recognition are more effective in recognition rate and safer A. The User Registration module for Android Client
than face recognition, nevertheless, their strict hardware
After the completion of offline feature extraction and
requirement and cumbersome operation prevent them being
feature index establishment, the user registration can be
popularized in people's daily lives. In contrast, face
recognition technology can be easily employed into mobile performed on Android mobile-phones. The steps are as
Apps, since it does not require hardware with high- follows.
configuration, but only needs a normal camera to shoot 1) Enter the registration page: Run the application to
human faces. enter its home page, click the Registration button to enter
Therefore, we combine the Android SDK and face the registration page.
recognition algorithm, and develop a mobile App for user 2) Registration: Fill some personal information in the
identity verification employing face recognition algorithm. registration page. The main information including user-
We implement two functions in our App: user registration defined username and gender needs to be collected.
and user log-in. Users could use the App to activate their

978-1-6654-0445-7/20/$31.00 ©2020 IEEE 288

DOI 10.1109/CIS52066.2020.00068

Authorized licensed use limited to: UNIVERSITY OF BATH. Downloaded on May 14,2021 at 09:53:43 UTC from IEEE Xplore. Restrictions apply.
 3) Completing face image collections: After the face image features, and then use the index structure for face
completion of collection of user information, click the Photo matching. Finally, the server send back the matching results
button to enter the photography page, in which, the user must to the mobile client, to complete the login process.
put his or her face into the pre-defined rectangular box to III. THE FACE RECOGNITION ALGORITHMS
collect at least 10 face images, to confirm an adequate
In terms of face recognition algorithm, it is the key step
diversity of faces. When operating, the user must click the
to extract good face features from human face images. In
camera button 10 times according to the application tips. general, face features can be divided into two categories:
After completing face image collections, the App will global features and local features [9]. The global feature is
automatically return to the registration page where the user that in each eigenvector, each eigen component contains all
could click the submit button. After receiving the success the information of the face image, and reflects the overall
acknowledgement from the server side, the App will jump to attributes of a face. The local feature reflects the information
the log-in completion page, to complete the whole of a certain part of a face image, presenting the details of the
registration procedures. various parts of the face.
In our mobile App we adopt the open source framework In our App, we tried three kinds of face recognition
HttpClient whose main function is encapsulation of HTTP algorithms, in which the Eigen [10] and Fisher [11]
messages. Users could encapsulate HTTP messages by recognition algorithm belongs to the global feature extraction
simply setting the submission content and address. After the algorithm, and the LBP [12] belongs to the local feature
completion of the photography, we could use HttpClient to extraction algorithm.
encapsulate the user's registration information into HTTP A. Eigen Face recognition algorithm
packets, and then submit them to the Web server where they
could be further processed. The Eigen Face recognition algorithm mainly uses the
dimension reduction method of PCA (principal component
B. The User login module for Android Client analysis)[13]. In this algorithm, a face image with a
The login module is resemble to the registration module, resolution of 100 * 100 can be formed a long eigenvector of
and requires several steps as well: 10,000 dimensions, which reduces the efficiency of
1) Enter the photography page : In the App home page, computation and storage. Therefore, the PCA dimension
click the camera button to enter the photography page. reduction method is introduced to extract the main
information of the original 10000 dimensional eigenvector,
2) Collect face images: In the photography page, locate
thus reduce the eigenvector dimensions. The following
user’s own face into the rectangular box and collect face shows the procedures:
images. 1) Calculate the mean vector μ for the eigenvector:
3) Login: After the system automatically returns to the
home page, the user clicks "login" button and the App waits X ^x1,x2 ,x3,...,xn `, where xi  R d ,
for the acknowledgement from sever side. If the App
received success acknowledgement, then it jumps onto the 
¦ [ L 
Q
welcome page. If received a denying message, the App will  μ 

ask the user to re-login. Q
Compared to the registration function, the login function
only need to collect one face image, therefore the login 2) Calculate the covariance matrix S:
process is much faster than the registration. We will discuss
the detailed efficiency analysis later in the experimental 1 n
analysis section.  S ¦ x  μ xi  μ T 
1 i

n
C. The Offline Feature Extraction Module
Before using the mobile application, it is necessary to
offline extract the eigenvalue from the face data set on the 3) Calculate the eigenvalue π i and the eigenvector vi :
server side. In order to choose the most appropriate face
recognition algorithm, in this application, we implement and  S * vi πi * vi  
compare three face image features. After image features
extracted, they are stored in an image feature library, and
then the method of Flann [8] is used to establish a linear 4) Decrease the eigenvalues:
index structure for these image features. The reason that we Decrease the eigenvalues so that the eigenvectors are
choose the linear structure is that it could save the feature consistent with the position of the eigenvalues. The larger the
without loss when the sample is small, and does not consume eigenvalue, the more significant the corresponding
too much query time when being queried. eigenvector is. Therefore, according to this principle, the
After the linear index is completely established, it will be eigenvector corresponding to the first k eigenvalues is
loaded when the server is started. When the server receives chosen as the eigenvector after dimension reduction. Thus,
the face image submitted by the client, it will first extract the the PCA dimension reduction process is completed.

289

Authorized licensed use limited to: UNIVERSITY OF BATH. Downloaded on May 14,2021 at 09:53:43 UTC from IEEE Xplore. Restrictions apply.
 This reduced eigenvector is the global eigenvector of the With 8 adjacent pixels surrounded, there can be 2 ^ 8
face image. For each face image, the feature is extracted and possible combinations, which is known as local binary
the linear feature is used for face feature matching. pattern (LBP). The first LBP operator described in the
literature employed a 3 * 3 neighborhood, whose feature
B. Fisher Face recognition algorithm extraction methodology is depicted as Fig. 2:
The above discussed PCA (principal component analysis)
removes some facial feature information when performing 1 2 2 0 0 0
dimension reduction, so the precision of face recognition will Threshold Binary: 00010011
be reduced if this feature information is significant. In 9 5 6 1 1 Decimal: 19

addition, because if the relatively low robustness of the PCA 5 3 1 1 0 0

algorithm, In the case of large changes in light, the PCA
Figure 2. LBP Feature Extraction Methodology
algorithm almost failed.
The goal of the Fisher Face algorithm is to maximize the A LBP operation can be defined as the following
dispersion between classes and minimize the dispersion equation:
within a class. In other words, to closer the feature distance
/%3 [ F \ F  ¦S
S 
between similar faces, and farth4er the feature distance of

 S VLS  LF  (9)
different faces. The specific algorithm is described as
follows: Where [F \ F  is the center pixel and LF is its
1) Make x a random vector from class c: brightness value, LS is the brightness value of its adjacent

 ; ^[ [ [ [F `
  

pixels, and s(•) is a symbolic
y function˖

(10)
 ;L ^[ [ [ [ Q `
  

IV. FACE DATA SET
In developing our App, we also collected a face dataset
2) Calculate the sum of the dispersion matrices: via sampling from the students aged 19 to 22 at Guilin
University of Aerospace Technology. A total of 133 student

¦ 1 L μ L  μ μ L  μ    faces were collected by Canon 5D Mark III SLR camera,
F 7
6%  with resolution set as 1920*1080. Each student was collected
with 10 to 20 face images, and each image requires
6
: ¦L ¦[
F
 M [ L
[ M  μ L [ M  μ L   
7 characters with different facial expressions. In addition, the
light intensity of each photo is made with some difference by
changing the exposure of the camera. This gives a feature
Where μ is the mean of all data, and μ_i is the internal diversity when extracting facial features, so that makes our
mean of a certain class. App effective in different photography scene.
3) Making the class of the largest separability: It can be seen from Table I that the face data set in this
paper has improved in terms of resolution, number of people
sampled and the total number of images compared to the
: 76%: Caltech dataset. In addition, Fig.3 shows that there are
:RSW DUJ PD[Z certain varieties in facial expressions and light intensity in
 : 7 6: :   collected images. It makes our data set effectively to validate
our implemented mobile App.
The next step is to solve this optimization problem and
TABLE I. ATTRIBUTES OF DATA SET
calculate the projection matrix W.
C. LBP recognition algorithm Dataset Resolution People Images

The LBP feature as a local feature of a texture has a very Caltech[14] 896*592 27 450
good effect in the field of face recognition. We also realized Ours 1920*1080 133 2,582
this algorithm, and compared it with Eigen Face, Fisher Face
which based on global features. The best face Recognition
algorithm is chosen from them, and finally employed in our
mobile application.
The basic idea of LBP is to sum the contrast results of
each pixel of the image and its local surrounding pixels.
Specifically, perform a threshold comparison for each pixel
(the center pixel) and its adjacent pixels. If a center pixel has
brightness greater than or equal to his adjacent pixel, it is
marked as 1, otherwise 0. Figure 3. Samples of Face Data Set

290

Authorized licensed use limited to: UNIVERSITY OF BATH. Downloaded on May 14,2021 at 09:53:43 UTC from IEEE Xplore. Restrictions apply.
 V. EXPERIMENT RESULTS AND ANALYSIS
A. Experiment Environment
Before discussing the experiment results and analysis,
our development and experiment environment shall be
introduced. The system is divided into two parts: the client
and the server. The following is the implementation
environment of the client:
x OS: Windows 7.
x Development Environment: Android studio 2.0,
Android SDK 4.1.
x Other open source framework: HttpClient 4.2.5.
B. Server Development Environment
x OS: Windows 7.
x Development environment: Eclipse Kepler, Web 2.5,
Tomcat 7.0.
x Computer vision library: OpenCV 2.4.6, JavaCV 0.1
x Other open source framework: HttpClient 4.2.5, Json
lib 2.4. Figure 4. Samples of Face Data Set
C. Comparison of Two Identification Method
It can be seen that the ROC curves of LBP feature (solid
In this paper, we mainly evaluate the face recognition line) have the largest coverage area and are nearly not
precision and the system efficiency. There are two ways to affected by resolution changes, so the LBP feature could be
evaluate the recognition precision, one is the overall considered the most effective; Similarly, the precision of
precision of the face data set, that is, the face recognition Eigen Face features (dashed line) hardly varies with
precision in a certain FAR (False accept rate); the other is the resolution changes; By contrast, the precision of Fisher Face
ROC curve (receiver operating characteristic curve) In features (dotted line) decreases as the resolution decreases,
addition, the system efficiency is evaluated by the time spent which means it is greatly influenced by the scale.
in every stage. For the sake of better user experience, the response time
TABLE II. COMPARISON OF FACE RECOGNITION PRECISION
for each stage of the application cannot be too long, so we
also examine the time efficiency of each stage of each
Precision rate @ FAR = 1% application module. The two main aspects, user operating
Method
100*100 200*200 300*300 time and system processing time are measured.
Eigen feature 34.31% 35.29% 32.35% The cost time for each stage of user registration module
is shown in Table III. Since the operation time for mobile
Fisher feature 21.57% 43.13% 37.25% phone client would not change as the resolution varies, we
Fisher feature 71.57% 76.47% 72.55% only tested the operation time in the resolution of 100*100,
instead of testing it in all resolutions. In Comparison, the
The first is the comparison of precision rate. Table II system processing time is affected by the resolution scales,
shows the precision of each face recognition method (FAR = because the file uploading time via the HTTP protocol rises
1%). In this experiment, taking into account the changes in as the file size increases. Therefore, the processing time in
image scale and the file transfer efficiency of Android client, each resolution is tested and compared, to choose the most
we employed three different image scales, whose resolution appropriate image resolution.
is 100 * 100, 200 * 200, and 300 * 300 respectively.
It can be seen from Table II that the LBP feature with a TABLE III. COST TIME FOR USER REGISTRATION
200*200 resolution achieves the highest precision.
Compared to the LBP feature, the Eigen Face feature is less Cost time
Phase
sensitive to the scale, and the Fisher Face feature is less 100*100 200*200 300*300
precise for low-resolution images. Therefore, when the Operation(s) 36.55 Ū Ū
system is implemented, we choose 200 * 200 resolution as
Processing(s) 4.87 5.18 5.67
standard image resolution for our system.
In order to comprehensively compare these three Total 41.42 41.73 42.22
algorithms, the ROC curve is also utilized to evaluate them Same as the user registration module, both the operation
with different resolutions. In Fig.4, (a) (b) (c) shows the time and processing time of the user login module were
ROC curves in the resolution of 300*300, 200*200 and tested, as shown in Table IV. By comparing the time spent in
200*200 respectively. the two modules, it seems that the login module is less time-
consuming than the registration module. The reason is that,

291

Authorized licensed use limited to: UNIVERSITY OF BATH. Downloaded on May 14,2021 at 09:53:43 UTC from IEEE Xplore. Restrictions apply.
in the login module only one face image is collected, while [1] Ye C. The development of smart phones on the mobile Internet tide[J].
in the registration module 10 images need to be gathered and Mobile Communications, 2012 (15): 12-16濁
processed to achieve a feature diversity. [2] Sun D, Qiu Z. An overview of biometrics [J]. Acta Electronica Sinica,
2001, 29(12): 1744-1748.
TABLE IV. COST TIME FOR USER LOGIN [3] Wang Y, et al. Identity verification based on iris recognition [J].Acta
Automatica Sinica, 2002. 28(1): p. 1-10.
Cost time [4] Jiang X, You X, Yuan Y, et al. A method using long digital straight
Phase
100*100 200*200 300*300 segments for fingerprint recognition [J]. Neurocomputing, 2012,
77(1): 28-35.
Operation(s) 2.88 Ū Ū
[5] Zhao W, Chellappa R, Phillips P J, et al. Face recognition: A
Processing(s) 0.103 0.149 0.191 literature survey[J]. ACM Computing Surveys, 2003, 35(4): 399-458.
[6] Zhang Y, Chen Q. The hybrid Achtecture of C/S and B/S[J].
Total 2.983 3.029 3.071 Computer Engineering and Applications, 2002, 38(23): 138-140.
Through the above comparison of the recognition [7] Hen-Tov A, Lorenz D H, Schachter L. Modeltalk: A framework for
precision and system processing time, we finally chose and developing domain specific executable models [J]. arXiv preprint
arXiv:0906.3423, 2009
employed the resolution of 200*200 in our system. With this
resolution, the face recognition could guarantee a relatively [8] Muja M, Lowe D. Scalable Nearest Neighbor Algorithms for High
Dimensional Data[J]. IEEE Transactions on Pattern Analysis and
high precision and time efficiency, and the images occupy Machine Intelligence, 2014, 36(11): 2227-2240.
less storage compared with those in 300*300 resolution. [9] Su Y, et al. Face recognition based on global and local feature
VI. CONCLUSION integration[J]. Journal of Software, 2010, 21(8): 1849-1862.
In order to solve the problem that users need to [10] Agarwal M, Jain N, Kumar M M, et al. Face recognition using eigen
remember multiple user accounts and passwords for different faces and artificial neural network[J]. International Journal of
mobile Apps, this paper implements an mobile App which Computer Theory and Engineering, 2010, 2(4): 624.
employs face recognition technology for user authentication, [11] Arandjelovic O, Shakhnarovich G, Fisher J, et al. Face recognition
and collects a dataset of human faces for testing. The with image sets using manifold density divergence[C]//IEEE
Computer Society Conference on Computer Vision and Pattern
experiment results show that our App achieves a good Recognition (CVPR). IEEE, 2005, 1: 581-588.
performance-users could successfully register or login via [12] Shyam R, Singh Y N. Face recognition using augmented local binary
face recognition in different light intensity and resolutions. pattern and Bray Curtis dissimilarity metric[C]//Signal Processing
and Integrated Networks (SPIN), 2015 2nd International Conference
ACKNOWLEDGMENT on. IEEE, 2015: 779-784.
This work was supported by Guangxi Young and [13] Abdi H, Williams L J. Principal component analysis[J]. Wiley
Interdisciplinary Reviews: Computational Statistics, 2010, 2(4): 433-
Middle-aged Teachers’ Basic Ability Improvement 459.
Foundation of China (No. 2020KY21023).
[14] Catech[DB] http://www.vision.caltech.edu/Image_Datasets/faces.tar
REFERENCES

Figure 1. The Application Framework.

292

Authorized licensed use limited to: UNIVERSITY OF BATH. Downloaded on May 14,2021 at 09:53:43 UTC from IEEE Xplore. Restrictions apply.