Reliable Face Identification System for Criminal

Investigation
2023 11th International Symposium on Digital Forensics and Security (ISDFS) | 979-8-3503-3698-6/23/$31.00 ©2023 IEEE | DOI: 10.1109/ISDFS58141.2023.10131738

Andrew Fredrick Nyoka Kelvin Sauli Godfrey Happines Gideon Mwakilembe Leonard Jonas Mugendi Oscar David Mbita
ICT Department ICT Department ICT Department ICT Department ICT Department
Tanzania Police Force Tanzania Police Force Tanzania Police Force Tanzania Police Force Tanzania Police Force
Dodoma, Tanzania Dodoma, Tanzania Dodoma, Tanzania Dodoma, Tanzania Dodoma, Tanzania
[email protected] [email protected] [email protected] [email protected] [email protected]

Adramane Assoumana Dawson Ladislaus Msongaleli

Institute for Basic Sciences, Technology and Innovation ICT Department
Jomo Kenyatta University of Agricultural and Technology Tanzania Police Force
Nairobi, Kenya Dodoma, Tanzania
[email protected] [email protected]

Abstract—Face identification technology has the potential to and comparing patterns of facial features. Emerging crimes
revolutionize criminal investigation by offering reliable and fast committed in digital space can be addressed by exploiting
ways of identifying criminals. Although there is significant digital solutions like facial recognition [2] [3]. In addition,
advancement in face recognition technology, adoption of this
technology in law enforcement agencies is at infancy because facial recognition technology has the potential to revolutionize
of factors like system reliability and complexity. In this study, criminal identification by providing law enforcement agencies
we propose a reliable face identification system (RFIS) for with a fast and reliable tool for identifying criminals [4].
criminal investigation; a novel multi-faced approach for facial Generally, facial recognition technology uses algorithms to
identification that identifies suspects by comparing live image, analyse various aspects of an individual’s face like the distance
still picture and video obtained from various sources. The
proposed system is developed by using two prominent libraries between the eyes, the curvature of the lips and the shape of the
called Dlib and Haar Cascade because of their effectiveness. nose to create a unique digital facial template [5]. Identification
In addition, RFIS uses Convolutional Neural Network (CNN) of individuals is conducted by comparing facial template with
architecture as the facial recognition algorithm which is trained a database of known faces. Facial recognition technology is
on a dataset of criminal facial images to learn the features now used in a wide range of applications, including security
that distinguish a criminal’s face from non-criminal. Notable
benefits of this study are increased reliability and accuracy in and surveillance, law enforcement, banking, and retail [6] - [8].
facial recognition system which is achieved through combining In law enforcement, facial recognition technology is used to
live picture recognition, still picture recognition and video image identify and track criminals, missing persons, and suspected
recognition. In addition, outstanding advantage of our solution terrorists. It is also used in border control and immigration
is addressing the potential for bias in the algorithm by using to verify the identity of individuals entering or leaving a
diverse dataset that includes faces of different ages, gender, races,
ethnicity and other characteristics. Furthermore, we explore the country. Moreover, software development environment can
effectiveness of our solution with the respect to Dlib and Haar affect performance of a facial recognition system significantly.
Cascade libraries and the results suggest that Haar Cascade Haar Cascade and Dlib libraries are common libraries used
library outperforms Dlib library in terms of execution time and in development of face recognition system. However, these
accuracy. libraries present several advantages and challenges [9]. In
Index Terms—Face Recognition, Artificial Intelligence, Ma-
chine Learning. [10], the authors present a comparison of two face recognition
techniques namely Haar Cascade and Local Binary Pattern.
I. I NTRODUCTION Results from their study suggest that Haar Cascade is more
accurate than Local Binary Pattern.
The history of facial recognition technology can be traced Furthermore, existing research publications address the
to the 1960s. Early facial recognition systems relied on man- quest of face recognition by considering live image recog-
ually measured distances between facial features to identify nition. In the context of criminal investigation, still picture or
individuals. Current systems use deep learning algorithms video footage may be used to identify a suspect. Thus, the
to learn and recognise faces automatically [1], thanks to quest for facial identification in criminal investigation must be
the recent development in artificial intelligence, software and addressed in a holistic approach; i.e., by considering live pic-
hardware. Facial recognition technology is an emerging bio- ture, still picture and video footage recognition. Accordingly,
metric technology that can identify individuals by analysing despite some recent advancement in facial identification, it
979-8-3503-3698-6/23/$31.00 © 2023 IEEE is still at infancy stages to be fully applied in law enforce-

Authorized licensed use limited to: CMR Institute of Technology. Downloaded on April 22,2025 at 09:06:58 UTC from IEEE Xplore. Restrictions apply.
ment agencies. Accuracy, reliability and complexity are key may involve comparing the features to a single image or to
challenges attributed to this situation [6]. Accuracy in facial multiple images from different angles and lighting conditions.
recognition systems is affected by several factors such as facial If a match is found, the system can identify the person in the
expression, pose, lighting etc. image or video feed hence returning the results and handling
In this study we present a robust and reliable facial identifi- the exceptions that might occur during the execution process.
cation system for criminal investigation. The proposed system It is worth noting that different facial recognition systems
takes into consideration three aspects of facial recognition may have variations in their structure and methods, and some
for criminal investigation, viz, live picture, still picture and systems may use additional steps such as image enhancement
video footage recognition. We explore the effectiveness of or machine learning algorithms to improve accuracy of the
libraries used in facial recognition by considering two promi- system.
nent libraries; Haar Cascade and Dlib libraries. Numerical Figure 2 shows the flowchart of RFIS by taking into consid-
results from our study are promising. The rest of the study eration the three aspects of our solution, i.e., live image, still
is organised as follows. Section II presents the working model picture and video identification. There are four stages in our
of RFIS. Section III shows a discussion of numerical results solution in the following order; user registration, crating facial
from our study and Section IV gives the conclusion of this dataset, training dataset and facial identification. Initially, all
research work. system users are registered in the system and then directed to
the login page of the system. Registered users are assigned
II. RFIS W ORKING M ODEL
special identification number which is mapped to the dataset
RFIS system uses a django framework that runs in a collected through a camera.
computer system with Ubuntu 23 Operating System. It uses
Logitech camera with high quality Full High Definition (FHD) Facial
1080P camera. Figure 1 presents a pictorial picture of the Recognition

proposed system. individual registration Create image dataset

Enter Id
Camera Image upload Video upload

Open camera

Matching process Train data

Trained Image
dataset dataset

Data base Data id & id

Fig. 1. RFIS Structure Personal ID

Return a page Personal ID
with details of
top Set of images containing
1. Full names matches ids
In order to detect an individual; the system scans an image 2. ( NIDA ) Number

or video feed and identifies regions that contain faces. In this 3. Nationality

case, Haar Cascade frontal face detection model which is an 4. Gender

5. Driving licence
xml file is used to detect faces before identification. Once
6. Other informations
a face is detected, the system aligns the face to a standard
pose and orientation, typically with the eyes, nose, and mouth Fig. 2. RFIS Flowchart
in the same relative positions. Note that, this is done after
the conversion of the image to grayscale. After detecting a We train collected dataset by initializing an instance of
face, the system extracts key features from the face, such as the Local Binary Pattern Histogram (LBPH) face recognizer
the distance between the eyes, the shape of the nose, and algorithm. We define a function that takes a directory path as
the curvature of the lips. These features are used to create input and returns two lists, a list of face images and a list of
a unique representation of the face which is stored in the corresponding identity. Initially, the function creates a list of
system. Face matching is done by comparing the extracted all image files in the directory using a list comprehension. It
features to a database of known faces to find a match. This then loops over each image file and does the following: opens

Authorized licensed use limited to: CMR Institute of Technology. Downloaded on April 22,2025 at 09:06:58 UTC from IEEE Xplore. Restrictions apply.
images files and converts to grayscale, converts grayscale Figure 3 shows that, when Dlib library is used, execution
image to a Numpy array, extracts the identity of the image time is proportional to the number of dataset employed pro-
from the file name, appends the face image and identity to the vided confidence remains constant. Results from Haar Cascade
faces and identities list respectively and display the face image. library suggest that execution time is not affected by the
After processing all images, the function returns a Numpy number of dataset. This is due to the fact that dataset collection
array of the identities and a list of the face images. After is done with care and professionally. Accordingly, there is a
training available datasets, we conduct facial identification by slight increase in execution time when the number of dataset
taking into consideration the following aspects; (i) real time is increased significantly, albeit with positive occurrence.
facial recognition (ii) still picture facial recognition and (iii)
video or CCTV footage facial recognition. In the first case; B. Accuracy
real time facial recognition, trained LBPH face recognizer Facial recognition technology has increasingly become ac-
is used to recognize faces and display a message once it is curate in recent years, thanks to advancements in machine
recognized. If the results are positive the system redirects the learning algorithms and data processing capabilities. However,
user to a details page of the recognized image. In the second the accuracy of facial recognition systems can vary depending
case; still picture recognition, a view function is defined in on several factors, including the quality of the images used
order to handle a POST request with an image file. Finally, for analysis, confidence, lighting conditions and the diversity
in video or CCTV footage facial recognition, a view function of the population being analyzed. Confidence in facial recog-
is defined to handle the upload of video file and perform face nition technology depends on several factors such as accuracy
recognition on the video. of the technology, the intended use case, ethical considerations
involved in its deployment, devices used like cameras, quality
III. N UMERICAL R ESULTS of images and distance between the camera and the face. In
this study, we express confidence as follows; (i) for real time
We present results of our study by considering the following facial recognition confidence ranges between 75% and 85%,
performance metrics; accuracy, execution time and number (ii) for still picture recognition confidence ranges between 75%
of dataset. We explore the effectiveness of our solution by and 85% and (iii) for video facial recognition, confidence lies
comparing the two prominent frameworks in facial recognition between 65% to 80%. These values depend on the quality of
namely Dlib and Haar Cascade. We run the simulation of image or video used. We express our results by considering
system on Computer with Ubuntu 23 Operating System, 16GB confidence against accuracy. Results in Figure 4 and Figure
RAM and Intel Core i7 Processor. Following is the discussion 5 are obtained after using Haar Cascade and Dlib Libraries
of results obtained from our simulation. In this context, we respectively.
keep confidence at 75%, an acceptable confidence in artificial
intelligence. 100

A. Execution Time
80
We explore the effect of number of dataset on our solution
by implementing it using Dlib and Haar Cascade library.
Accuracy (%)

60
30

40
Execution Time (Secoonds)

20 20 Still Picture
Real Time
Video
0
20 40 60 80 100
10
Confidence (%)

Dlib Library Fig. 4. Confidence vs. Accuracy based on Haar Cascade Library
Haar Cascade Library
0 Results from our study suggest that, Haar Cascade Library
40 60 80 100 120 140 160 180 is more accurate compared to Dlib Library. In addition,
Number of Dataset increasing confidence does not affect accuracy significantly
when Dlib library is used. The main reason for low accuracy
Fig. 3. Number of Dataset vs. Execution Time when Dlib libray is used is that it uses coloured image to

Authorized licensed use limited to: CMR Institute of Technology. Downloaded on April 22,2025 at 09:06:58 UTC from IEEE Xplore. Restrictions apply.
train dataset, accordingly, images have a lot of features which mation Fidelity (VIF). We express our results by considering
reduce the accuracy of the system. quality of image expressed in PSNR against execution time.
This is due to the fact that PSNR is mostly used to measure
100 the quality of image [11] - [13]. In this context, execution
time refers to the time taken by each compression algorithm to
compress the image to the specified quality level. Results from
80 our study suggest that Haar Cascade library is not affected by
quality of image while Dlib library is significantly affected by
quality of image used as shown in Figure 6. Haar Cascade
Accuracy (%)

60 Library is not significantly affected by quality of image as

shown in Figure 6. Results from our study in Figure 7 suggest
that, Dlib library is affected by quality of image, i.e., execution
40 time increases as quality of image increases.

20 Still Picture
Real time
Video
2.5 Image
Real Time Video
0
20 40 60 80 100

Execution Time (Seconds)

2

Confidence (%)
1.5
Fig. 5. Confidence vs. Accuracy based on Dlib Library

1
C. Quality of Image
There are several units that can be used to measure the 0.5
quality of an image. One commonly used unit is the Peak
Signal-to-Noise Ratio (PSNR). It measures the difference
between the original image and the compressed or distorted 15 30 45 60 75
image. PSNR is expressed in decibels (dB) and is calculated Quality of Image (PSNR (dB))

as:
M AX 2 Fig. 7. Quality of image vs. Execution Time based on Dlib Library
 
P SN R = 10log , (1)
M SE
where MAX is the Maximum Possible Pixel value (e.g., 255
for an 8-bit image), and MSE is the Mean Squared Error IV. C ONCLUSION
between the original and compressed images [11].
This study presents a novel face recognition system
real time for criminal investigation. The proposed system identifies
2.5 image suspects by comparing live image, still picture and video
video obtained from various sources. RFIS is developed by using
two prominent libraries called Dlib and Haar Cascade because
Execution Time (Seconds)

2
of their effectiveness. In addition, RFIS uses Convolutional
Neural Network (CNN) architecture as the facial recognition
1.5
algorithm which is trained on a dataset of criminal facial
images to learn the features that distinguish a criminal’s
1 face from non-criminal. We express results of our study
by considering execution time, confidence, accuracy and
0.5 quality of image. Furthermore, a comparison of Haar Cascade
and Dlib library is conducted. Numerical results obtained
from simulations suggest that Haar Cascade library is more
15 30 45 60 75 effective than Dlib library. Notable benefits of this study are
Quality of Image (PSNR(dB))
increased reliability and accuracy in facial recognition system
Fig. 6. Quality of image vs. Execution Time based on Haar Cascade Library which is achieved through combining live picture recognition,
still picture recognition and video image recognition.
Other measures of quality of image are Structural Similarity
Index (SSI), Mean Opinion Score (MOS) and the Visual Infor-

Authorized licensed use limited to: CMR Institute of Technology. Downloaded on April 22,2025 at 09:06:58 UTC from IEEE Xplore. Restrictions apply.
 R EFERENCES
[1] A. Kumar, A. Kaur, and M. Kumar, “Face Detection Techniques: A
review,” Artificial Intelligence Review, 2019; 52 (2): 927- 948.
[2] D. L. Msongaleli and I. P. Mgembe, “Three-Tier Database Forensic
Model,” International Symposium on Digital Forensics and Security
(ISDFS), Istanbul, Turkey, 2022.
[3] I. P. Mgembe, D. L. Msongaleli and N. K. Chaundhary, “Progressive
Standard Operating Procedures for Darkweb Forensics Investigation,”
International Symposium on Digital Forensics and Security (ISDFS),
Istanbul, Turkey, 2022.
[4] M. A. Lazarini, R. Rossi, and K. Hirama, “A Systematic Literature Review
on the Accuracy of Face Recognition Algorithms”, EAI Endorsed Trans
IoT, vol. 8, no. 30, p. e5, Sep. 2022.
[5] A. Tolba, A. El-Baz, and A. El-Harby, “Face Recognition: A Literature
Review,” International Journal of Signal Processing, pp. 88-103, 2005.
[6] C. Kanan, and G. W. Cottrell “Color-to-Grayscale: Does the Method
Matter in Image Recognition?,” PLoS ONE 7(1): e29740., 2012,
doi:10.1371/journal.pone.0029740
[7] P. R. Patil, and S. S. Kulkarni, “Survey of Non-intrusive Face Spoof
Detection Methods,” Multimedia Tools and Applications, 2021; 80 (10):
14693-14721.
[8] Y. Kortli, M. Jridi, A. Al Falou, and M. Atri, “Face Recognition Systems:
A Survey,” Sensors, 2020; 20 (2): 342.
[9] N. Ibrahim, Y. Yusof and M. G. Bin Ithnin, “Haar Cascade Vs Facial
Landmarks Technique for Face Recognition Uniqueness,” International
Journal of Advanced Trends in Computer Science and Engineering, vol
8. 2019, pp. 525-530.
[10] A. B. Shetty et. al., “Facial recognition using Haar cascade and LBP
classifiers,” Global Transitions Proceedings, vol. 2, Issue 2, 2021, pp.
330-335.
[11] O. S. Faragallah et al., “A Comprehensive Survey Analysis for Present
Solutions of Medical Image Fusion and Future Directions,” IEEE Access,
vol. 9, pp. 11358-11371, 2021, doi: 10.1109/ACCESS.2020.3048315.
[12] S. A. Elmasry, W. A. Awad, and S. A. Abd El-hafeez, “Review of
different image fusion techniques: Comparative study,” Internet of Things
Applications and Future. Singapore: Springer, 2020, pp. 41–51.
[13] D. H. Xia, S. Song, L. Tao, Z. Qin, Z. Wu, Z. Gao, J. Wang, W.
Hu, Y. Behnamian, and J.-L. Luo, “Review-material degradation assessed
by digital image processing: Fundamentals, progresses, and challenges,”
Journal of Material Science and Technology, vol. 53, pp. 146–162, Sep.
2020.

Authorized licensed use limited to: CMR Institute of Technology. Downloaded on April 22,2025 at 09:06:58 UTC from IEEE Xplore. Restrictions apply.