Comparing the Effectiveness and Performance of

Image Processing Algorithms in Face Recognition

Hussam M. Rostum József Vásárhelyi
Institute of Automation and Institute of Automation and
Info-Communication Info-Communication
University of Miskolc University of Miskolc
Miskolc, Hungary Miskolc, Hungary
orcid: 0000-0002-1630-248X orcid: 0000-0002-7328-6877

Abstract—Many institutions have recently embraced bio- emerged as a potent and effective educational tool for tackling
metric security solutions, utilizing biological measurements to computer vision tasks, including those associated with facial
2024 25th International Carpathian Control Conference (ICCC) | 979-8-3503-5070-8/24/$31.00 ©2024 IEEE | DOI: 10.1109/ICCC62069.2024.10569864

safeguard against fraudulent activities, theft, and various security recognition [4]. The remarkable progress in the field of
threats. Face recognition technology holds a pivotal role within
the realm of bio-metric security systems, serving purposes such as information technology has propelled deep learning to the
authentication, monitoring, individual identification, and identity forefront as a pivotal artificial intelligence technique for image
verification. This article aims to delve into the examination of processing [5]. Deep learning has proven its mettle by
facial recognition systems grounded in deep learning. This focus delivering exceptional outcomes, particularly in tasks like pre-
arises due to the intricate nature of the process and the existence processing (e.g., direction and lighting correction) and face
of numerous hurdles and variables that impact algorithm perfor-
mance. The objective here is to illuminate the foremost challenges recognition. The success of deep learning models can be
that real-world systems encounter, often overlooked in previous largely attributed to the ready availability of potent tools and
research. Additionally,under these challenges, the article will computational resources, such as Graphics Processing Units
conduct a comparative analysis of the performance of prominent (GPUs). Furthermore, the abundance of extensive datasets
facial recognition algorithms, namely VGGFace, FaceNet, and plays a crucial role [6]. Deep learning algorithms excel at
ArcFace. This academic approach will allow to make informed
choices when selecting the most suitable algorithms for specific automatically extracting vital features from image data in
applications. a hierarchical fashion, operating at multiple levels [7] In
Index Terms—Deep Learning, Facial Recognition, Convolu- scenarios where identifying a person is impeded by obstacles,
tional Neural Networks, VggFace, ArcFace, FaceNet, Cosine deep learning algorithms encounter challenges in discerning
Similarity, Euclidean Distance distinctive features for identity verification. As a response,
scientists have diligently worked to devise numerous deep
I. I NTRODUCTION learning models for facial recognition, incorporating strategies
Facial recognition technology stands out as a pivotal tool that involve convolutional neural networks (CNNs) and various
for identifying individuals based on digital images. It holds a other innovative approaches. Consequently, it has become im-
prominent position in the realm of bio-metric measurements perative to gauge the performance of deep learning techniques
due to its distinctive features compared to other methods like in the realm of face recognition across diverse conditions
iris recognition and fingerprint scanning. Unlike these alter- and circumstances [8]. This research will specifically center
natives, which require direct contact with the person, facial its attention on the VGGFace [9], [4], FaceNet [10],
recognition operates without such constraints. Additionally, and ArcFace [11] algorithms. To assess their performance,
its cost-effectiveness has positioned it as a leader in the we employed a collection of images sourced from the Yale
domains of security and surveillance [1] [2]. Nonetheless, database and introduced a range of supplementary evaluation
the effectiveness of the face recognition system is significantly criteria, enabling us to measure the efficacy of these algorithms
influenced by a multitude of factors, making it a paramount in a variety of real-world operational settings, different from
challenge for researchers in the realm of computer vision. conventional test conditions. This research will cover the basic
Consequently, efforts have been directed towards enhancing algorithms for face recognition. The third section defines the
conventional techniques that involve manual feature extraction. comparison parameters, and the fourth section discusses the
These methods rely on traditional machine learning tools like results of testing the three algorithms.
the k-Nearest Neighbors algorithm (KNN), Support Vector
Machines (SVM), and others to carry out the task of facial II. VGGFACE M ODEL
recognition and analysis [1], [3]. In the contemporary
era, traditional methods have proven inadequate in address- VGGFace alludes to a series of algorithms developed by re-
ing the myriad challenges posed by facial recognition. As searchers affiliated with the University of Oxford, specifically
a response, Convolutional Neural Networks (CNNs) have by the University’s Visual Geometry Group (VGG). These

979-8-3503-5070-8/24$31.00 ©2024 IEEE

Authorized licensed use limited to: BME OMIKK. Downloaded on March 28,2025 at 11:57:43 UTC from IEEE Xplore. Restrictions apply.
algorithms consider the most frequently employed diagnostic
assessments in their design.
This version was crafted in 2015 and was constructed upon
the VGG16 model [4], using a 16-layer convolutional neural
network. It underwent training using a dataset of 2.6 million
images associated with 2,622 individuals. Notably, this model
achieved an accuracy of 98.95% when assessed using the
LFW dataset [11]. VGGFace features an internal architecture
Fig. 2. Structure of FaceNet Algorithm
comprising 38 layers, and it is designed to process images
with a dimension of 224 x 224 pixels. Fig.1 represents a visual
representation of the VGGFace algorithm’s structure.

Fig. 3. Triplet loss

and recognition [12]. ArcFace operates in the following

manner: it takes two facial images as input and produces a
distance metric between them, assessing the likelihood that
Fig. 1. VGGFace algorithm’s structure
they belong to the same person. This distance is computed
using the Cosine Distance, a method also employed by search
III. FACE N ET M ODEL engines [13]. The Cosine Distance can also be computed
FaceNet was introduced by researchers from Google in through the dot product of two vectors, resulting in a situation
2015, as detailed in their article titled ”A Unified Em- where, if the two vectors are identical, Θ = 0, and Cos(Θ)=1.
bedding for Face Recognition and Clustering” [6]. In their Conversely, if the vectors are perpendicular, Θ = π/2, and
approach, they developed high-quality facial representations Cos(Θ)= 0.
from images, employing various deep-learning architectures, Consequently, this Cosine Distance serves as a means to
like Inception and ZF-Net. FaceNet operates by taking a gauge similarity, as depicted in Fig.5. The accuracy of the
facial image as input and producing a vector composed of ArcFace algorithm reached 98.01% when evaluated using the
128 numerical values, which encapsulate the most crucial LFW dataset [11].
facial characteristics. This vector is commonly referred to as
an ”embedding”. Fig.2 shows the structure of this model.
Subsequently, the researchers employed a technique known
as ”triplet loss” as a loss function to train this architecture,
illustrated in Fig.3 . Triplet loss aims to minimize the distance
between the reference image and the positive (matching)
image, while simultaneously maximizing the distance between
the reference image and the negative images. Fig. 4. The structure of ArcFace algorithm
FaceNet’s training data encompassed an impressive range,
involving approximately 100 million to 200 million thumbnail
images representing around 8 million distinct identities. As a
testament to its performance, FaceNet achieved an accuracy
rate of about 99.63% when evaluated using the LFW dataset
[11]. The model is designed to accommodate input images
with dimensions ranging from 96 x 96 pixels to 224 x 224
pixels.
IV. A RC FACE M ODEL
ArcFace, also known as Additive Angular Margin Loss or Fig. 5. Calculate the distance by using the cos θ
Adaptive Angular Margin Loss, as illustrated in Fig.4, is a
deep learning model constructed on the ResNet100 architec- Table. I shows the summary of predictions between the
ture. This model is applicable for tasks such as face detection previous three algorithms.

Authorized licensed use limited to: BME OMIKK. Downloaded on March 28,2025 at 11:57:43 UTC from IEEE Xplore. Restrictions apply.
 TABLE I • Face Expressions: Determine the ability of algorithms to
SHOWS A COMPARISON OF THE ALGORITHMS USED recognize people when their facial expressions change
Algorithm VGGFace FaceNet ArcFace [1].
Num of Test Images 2.6M 100M-200M 11.96M • Different lighting conditions: Determine the ability of
Num of Persons 2622 8M - algorithms to distinguish people in different lighting
Image Dim 224x224 96x96-224x224 224x224
Layers Num 38 22 100 conditions [17].
Accuracy • The presence of obstacles on the face, such as wearing
98.95% 99.63% 98.01%
LFW glasses [18], [19].
This aims to determine the ability of algorithms to recognize
faces in more realistic working conditions, taking into account
V. C OMPARISON PARAMETERS
the effect of the distance function on accuracy.
This research conducted a comparative analysis of the Table II displays the errors arising from wearing glasses for
algorithms focusing on their accuracy in face recognition. the three algorithms, whereas Table III illustrates the errors
Accuracy, in this context, is defined as the degree of proximity stemming from the impact of the lighting factor. Additionally,
to the true value or, put differently, the level of fidelity. Table IV outlines the influence of facial expressions on the
Additionally, the research delved into examining the impact of performance of these algorithms.
the distance function on accuracy. The distance function serves
as a measure of similarity and proximity between vectors TABLE II
extracted from images. In the analysis, the research selected E RRORS RESULTING FROM ALGORITHMS WHILE WEARING GLASSES
three distinct distance functions: Distance fanctions ArcFace FaceNet VGGFace
• Equation (1) shows the cosine similarity: [14]. Cosine distance 100% 100% 100%
Euclidean distance 100% 100% 99.37%
AB Euclidean L2 distance 100% 100% 100%
Similarity = cos θ = , (1)
∥A∥ ∥B∥
It is observed that the VGGFace algorithm exhibits height-
where A, B are non-zero vectors. ”Cosine similarity is ened sensitivity to obstacles in front of the face. Its accuracy
a measure of similarity between two non-zero vectors of reaches 99.37% when tested for sensitivity to the presence
inner product space” of glasses using the Euclidean distance function. In contrast,
• Euclidean distance with the equation (2) [15]: the remaining algorithms successfully overcome this challenge
v
u n without encountering any errors.
uX
∥p − q∥ = t (pi − qi )2 ) (2)
i=1 TABLE III
E RRORS GENERATED BY ALGORITHMS IN DIFFERENT LIGHTING
CONDITIONS
where qi , pi represent Euclidean vectors, starting from the
origin of the space (initial point), and n is the n-space Distance fanctions ArcFace FaceNet VGGFace
• Squared Euclidean L2 distance with the equation is: Cosine distance 98.13% 98.76% 98.13%
Euclidean distance 98.13% 98.75% 98.11%
n
X Euclidean L2 distance 98.13% 97.47% 98.13%
∥p − q∥ = (pi − qi )2 (3)
i=1 It is worth highlighting that the FaceNet algorithm stands
where qi , pi represent the Euclidean vectors, starting from out as the most proficient choice for operating in diverse light-
the origin of the space (initial point), n is the n-space. ing conditions. It demonstrates the highest accuracy among
Equation (3) shows the Squared Euclidean distance where the algorithms, as indicated by both the Euclidean distance
qi , pi represent Euclidean vectors, starting from the origin function and the Cosine function. Given this observation, the
of the space (initial point).And n is the n-space [15]. FaceNet algorithm emerges as a preferred option for surveil-
lance systems operating in challenging lighting conditions,
VI. T HE PROPOSED METHOD AND THE COMPARISON such as parks and roads.
RESULTS

This section will compare the three previously discussed TABLE IV

E RRORS RESULTING FROM ALGORITHMS DUE TO DIFFERENT FACIAL
models under various conditions using the Yale face dataset EXPRESSIONS
[16]. The database is divided into multiple groups, such as
center lighting, face with glasses, happy expressions, left light- Distance fanctions ArcFace FaceNet VGGFace
Cosine distance 98.75% 98.76% 98.75%
ing, face without glasses, normal conditions, right lighting, Euclidean distance 98.75% 98.13% 98.11%
sad expressions, sleepy appearances, surprised expressions, Euclidean L2 distance 98.75% 98.11% 98.75%
and winking instances. The research present an innovative
approach to assess algorithm sensitivity to the following Similar performance is observed among the three algorithms
working conditions: when assessing their sensitivity to various facial expressions.

Authorized licensed use limited to: BME OMIKK. Downloaded on March 28,2025 at 11:57:43 UTC from IEEE Xplore. Restrictions apply.
However, the stability exhibited by ArcFace positions it as the VGGFace, FaceNet, and ArcFace. To assess their accuracy in
most suitable for operating in environments characterized by real world scenarios, the research utilized the Yale database
good lighting conditions and diverse facial expressions, such as as a benchmark for evaluation. Upon comparing the outcomes
airports and bus stations. In conclusion, the research calculated obtained from independently studied algorithms using the
the accuracy of the algorithms, considering their sensitivity to LFW dataset, it becomes evident that these individual
the aforementioned factors. Notably, it observed variations in networks exhibit variations in their performance when faced
accuracy corresponding to changes in the distance factor used with obstacles. The data can be refined in distinct ways, as
in the calculation. Table V provides a detailed overview of the evidenced by subsequent works that leverage broader and
accuracy of these algorithms based on the preceding results. more comprehensive datasets. Further discussion is warranted
to delve into the nuances of these observations and refine
TABLE V the focus for future research endeavors. Future research can
ACCURACY OF THE STUDIED MODELS explore the use of more diverse and extensive datasets beyond
Distance fanctions ArcFace FaceNet VGGFace the Yale database. Including datasets with a broader range
Cosine distance 96% 97% 96% of facial expressions, lighting conditions, and demographics
Euclidean distance 96% 96% 95% would provide a more comprehensive evaluation of the
Euclidean L2 distance 96% 95% 96%
models’ performance in real-world scenarios.
These tables reveal that the accuracy of the ArcFace al-
gorithm remains constant and unaffected by changes in the
A BBREVIATIONS
distance factor used in the calculation, maintaining an accuracy
of approximately 96%. In contrast, the FaceNet algorithm is The following abbreviations are used in this manuscript:
notably influenced by distance functions, showing its highest
accuracy at 97% with the cosine distance and its lowest KNN k-Nearest Neighbors algorithm
accuracy with the Euclidean distance L2 . This sensitivity is at- SVM Support Vector Machine
tributed to its heightened responsiveness to lighting conditions. CNN Convolutional Neural Networks
As for the VGGFace algorithm, its accuracy remains consistent GPUs Graphics Processing Units
when utilizing both the cosine distance and the Euclidean VGGFace Visual Geometry Group Face
distance L2 , estimated at 96%. However, this accuracy slightly LFW Labeled Faces in the Wild
decreases to 95% when employing the Euclidean distance, ResNet Residual Network
indicating the algorithm’s sensitivity to the presence of glasses.
Table VI presents a practical comparison among the three R EFERENCES
algorithms, where the research evaluated their accuracy using [1] Montero, D., Nieto, M., Leskovsky, P. and Aginako, N., 2022, October.
both the LFW dataset and the method proposed in this Boosting Masked Face Recognition with Multi-task Arcface. In 2022
research. It is noteworthy that the Cosine distance function 16th International Conference on Signal-Image Technology & Internet-
Based Systems (SITIS) (pp. 184-189). IEEE.
consistently yields the best performance for the algorithms [2] Kumarapu, L., Dubey, S.R., Mukherjee, S., Mohan, P., Vinnakoti,
under various working conditions. Consequently, the research S.P. and Karthikeya, S., 2023. WSD: Wild Selfie Dataset for Face
has employed the values obtained using the Cosine distance Recognition in Selfie Images. arXiv preprint arXiv:2302.07245.
[3] DHARMADINATA, O.J. and KUSUMA, G.P., 2023. Improving Face
function in the subsequent table. Recognition in Low Illumination Condition Using Combination of Image
Enhancement and Face Recognition Methods. Journal of Theoretical and
TABLE VI Applied Information Technology, 101(4).
C OMPARISON OF ALGORITHM ACCURACY USING OUR PROPOSED METHOD [4] Goel, R., Mehmood, I. and Ugail, H., 2021. A Study of Deep Learning-
AND LFW DATASET based Face Recognition Models for Sibling Identification. Sensors,
21(15), p.5068.
ArcFace FaceNet VGGFace [5] Shahsavarani, S., Analoui, M. and Ghiass, R.S., 2020. M2̂ Deep-ID: A
Precision in LFW 98.01% 99.63% 98.95% Novel Model for Multi-View Face Identification Using Convolutional
Precision when Deep Neural Networks. arXiv preprint arXiv:2001.07871.
100% 100% 100% [6] Schroff, F., Kalenichenko, D. and Philbin, J., 2015. Facenet: A Unified
wearing glasses
Accuracy in different Embedding for Face Recognition and Clustering. In Proceedings of the
98.13% 98.76% 98.13% IEEE conference on computer vision and pattern recognition (pp. 815-
lighting conditions
Accuracy when 823).
98.75% 98.76% 98.75% [7] Serengil, S.I. and Ozpinar, A., 2020, October. Lightface: A Hybrid Deep
expressions differ
Average accuracy of Face Recognition Framework. In 2020 Innovations in Intelligent Systems
96% 97% 96% and Applications Conference (ASYU) (pp. 1-5). IEEE.
the proposed method
[8] Prasad, P.S., Pathak, R., Gunjan, V.K. and Ramana Rao, H.V., 2020.
Deep learning based representation for face recognition. In ICCCE 2019:
VII. C ONCLUSION Proceedings of the 2nd International Conference on Communications
and Cyber Physical Engineering (pp. 419-424). Springer Singapore.
This investigation conducted a performance comparison [9] Mandal, B., Okeukwu, A. and Theis, Y., 2021. Masked face recognition
of contemporary deep learning models within the context using resnet-50. arXiv preprint arXiv:2104.08997.
[10] Chandra, Y.B. and Reddy, G.K., 2020. A comparative analysis of face
of realistic working conditions. The innovative approach recognition models on masked faces. International Journal of Scientific
involved the comparison of three pre-trained models & Technology Research, 9(10), pp.30175-0.

Authorized licensed use limited to: BME OMIKK. Downloaded on March 28,2025 at 11:57:43 UTC from IEEE Xplore. Restrictions apply.
[11] Ranjan, R., Bansal, A., Zheng, J., Xu, H., Gleason, J., Lu, B., Nanduri, Accessed: 17 January 2024. Available: https://en.wikipedia.org/wiki/
A., Chen, J.C., Castillo, C.D. and Chellappa, R., 2019. A fast and Euclidean distance
accurate system for face detection, identification, and verification. IEEE [16] Oloyede, M.O., Hancke, G.P. and Myburgh, H.C., 2020. A review on
Transactions on Biometrics, Behavior, and Identity Science, 1(2), pp.82- face recognition systems: recent approaches and challenges. Multimedia
96. Tools and Applications, 79, pp.27891-27922.
[12] Deng, J., Guo, J., Xue, N. and Zafeiriou, S., 2019. Arcface: Additive [17] Anwarul, S. and Dahiya, S., 2020. A comprehensive review on face
angular margin loss for deep face recognition. In Proceedings of the recognition methods and factors affecting facial recognition accuracy.
IEEE/CVF conference on computer vision and pattern recognition (pp. Proceedings of ICRIC 2019: Recent Innovations in Computing, pp.495-
4690-4699). 514.
[13] Parkhi, O., Vedaldi, A. and Zisserman, A., 2015. Deep face recognition. [18] Lin, S.H., 2000. An introduction to face recognition technology. Inform-
In BMVC 2015-Proceedings of the British Machine Vision Conference ing Sci. Int. J. an Emerg. Transdiscipl., 3, pp.1-7.
2015. British Machine Vision Association. [19] Dang, T.V., 2022. Smart home Management System with Face Recog-
[14] “Cosine similarity” Wikipedia. Accessed: 15 February 2024. Available: nition based on ArcFace model in Deep Convolutional Neural Network.
https://en.wikipedia.org/wiki/Cosine similarity Journal of Robotics and Control (JRC), 3(6), pp.754-761.
[15] “Euclidean distance and Squared Euclidean distance” Wikipedia.

Authorized licensed use limited to: BME OMIKK. Downloaded on March 28,2025 at 11:57:43 UTC from IEEE Xplore. Restrictions apply.