ISBN 978-1-6654-3986-2

Real Time Crime Detection Using Deep Learning

Algorithm
2021 International Conference on System, Computation, Automation and Networking (ICSCAN) | 978-1-6654-3986-2/21/$31.00 ©2021 IEEE | DOI: 10.1109/ICSCAN53069.2021.9526393

P. Sivakumar Jayabalaguru. V
Professor, Department of Information Technology, Department of Information Technology,
Manakula Vinaygar Institute of Technology, Manakula Vinaygar Institute of Technology,
Pondicherry, India. Pondicherry, India.
[email protected] [email protected]

Ramsugumar. R. Kalaisriram. S
Department of Information Technology, Department of Information Technology,
Manakula Vinaygar Institute of Technology, Manakula Vinaygar Institute of Technology,
Pondicherry, India. Pondicherry, India.
[email protected] [email protected]

Abstract - The Crime Rate and number of Criminals are increasing

day by day, so there is great concern about the security issues, so II. NEED OF THE SYSTEM
to prevent and identify the crime before it occurs is the primary
goal of the police officials. With the help of Recent Technologies, As we know that prevention is better than cure, preventing a
especially CCTV is normally deployed in every private and public crime before occurring is better than investigating what or how
area to control crime but it needs human supervision to monitor. the crime had occurred. Just like vaccination is given to people
It's hard for a human to monitor many screens at the same time. to prevent the disease, in today’s world with a way higher rate
It leads to many errors. To overcome these problems, we proposed and crime happenings, it's become necessary to possess a Crime
Real-Time Crime Detection Technique using a Deep Learning
detection technique that prevents the crime happenings. These
Algorithm which monitors real-time videos and alerts the nearby
Cybercrime admin about the occurrence of crime with current Detection Techniques can be used by police officials to detect
location. In this paper, We present YOLO as our object detection the crime before it occurs so that the crime can be prevented.
algorithm. Our architecture is extremely fast and process image in
real-time at 45 frames per second.
III. RELATED WORKS
Keywords — Deep learning; YOLO; Object Detection; Mean
Average Precision.
A. VIDEO SURVEILLANCE
I. INTRODUCTION
Pratibha, Akanksha, and Gahalot presented an automated
YOLO is a deep learning object detection algorithm video surveillance system using statistical characteristics for
that uses a Convolution neural network, which is one of the crime detection. If there is any kind of abnormal situation like
most effective algorithms. It differs from the rest of the kidnapping, chain snatching, and fighting, the trained model
algorithms because it uses a single convolution neural network. alerts the situation to the nearby police station. Predicting the
Bounding boxes and their corresponding probabilities are human behavior and moving target are the key features of the
predicted by the One Convolutional neural network. The trained existing system. There are many advanced cameras and CCTV
model makes its detection based on its final weights trained by infrastructure installed in many areas. The main goal of the
the dataset so that the performance of the trained model can be crime video surveillance system is to provide automatic
maximized. When the system is developed, there are many detection of the crime scene by analyzing the movements for
challenges faced in developing the model. To use YOLO preventing future incidents[11].
algorithm, we need maximum utilization of the Graphical
Processing Unit, so we use Google Colaboratory which comes
with an in-built Tesla k80 GPU, Which is a more powerful and B. WEAPON DETECTION
faster GPU. To increase the Mean Average Precision and to
decrease the final average loss of the model, the database which Weapon detection in real-time image and video is already
is used to train the model must be bigger. The Trained model is existing research. We go through several image classification
able to detect many images even in a complex scene. research including analyzing data from various weapons, then
detection of violent crime scenes and detection of a gun in the

Manakula Vinayagar Institute of Technology

Puducherry,
Authorized India
licensed use limited to: Vignana Bharathi Institute Of Technology. Downloaded on July 13,2024 at 07:12:49 UTC fromIEEE ICSCAN
IEEE Xplore. 2021apply.
Restrictions
 ISBN 978-1-6654-3986-2

video feed. Most of the research was not implemented in deep

learning/neural network techniques so that this paper focuses
V. PROPOSED TECHNIQUES
on automatic detection of the gun in surveillance camera and
alert if the gun is detected[4].
Crime is increasing considerably day by day. With the rapid
increase in violent crimes, an efficient surveillance system has
C. OBJECT DETECTION
become necessary. In this, a model has been proposed for
detecting face and object when a criminal with threatening
Nowadays, In the field of Computer vision object weapons was caught under the surveillance of the CCTV
detection plays a major role. The main objective of object camera. The server is connected with a real-time CCTV camera
detection is to detect and classify objects in several fields and at each location. The criminal face and weapons are stored in
applications like face detection and recognition. Our model the database which is connected to the server. The database is
success is based on the fast object detection algorithm and its maintained by the server that contains all weapons and
efficiency. In recent times, the attention towards object criminals to be found. Live feed obtained from the CCTV
detection is increasing, Several Real-time applications like camera, if the Criminal is present in any feed, then the live
security applications, automatic driving, robotics, etc are location of that crime will be tracked and sent to the cybercrime
working under object detection. Computer vision and image admin. The Live CCTV image is trained with the existing
processing are associated with object detection technology with trained dataset.
deals with digital images and videos[12].

D. YOLO

Detection and classification of object detection is the main

purpose of the You Only Look Once algorithm. There are four
versions in the YOLO algorithm, each version will improve the
performance of the model. When comparison with RCNN, the
YOLO algorithm uses the feature of the entire image rather than
a part of the captured image. YOLO algorithm performs
bounding box and class prediction are done simultaneously, so
it is unique from other traditional systems. YOLO treats the
object detection problem as a regression problem instead of
having a normal region proposal and classification. This makes
YOLO perform more efficiently in real-time with some loss of
accuracy.[8]

IV. EXISTING TECHNIQUES Fig.1. Architecture Diagram for proposed system

Apporva.P, Impana H.C presented a real-time automated A. Dataset

criminal detection system using Haar feature-based cascade
classifier. This system will be very much helpful to detect and Creating a Dataset of Many High-Resolution images of our
recognize the criminal face automatically in real-time. desired classes and objects. The data we collected as criminals
consists of 100 images for each criminal, and also we collected
The existing technique for criminal detection consists of 4 Images of Weapons like a knife, gun, piston, etc. Criminal Face
steps: 1. Training the Criminal Images, 2. Face Detection using Detection in this dataset is very challenging because of rich
Haar cascade algorithm, 3. Compare the trained images with an variations in pose, scale, face expressions, etc. To create the
image captured from the real-time surveillance camera, 4. dataset we are using the ybat annotation tool for annotating and
Result Comparison whether Criminal or not. labeling the images. The boxes must be already be manually
annotated to ensure precision and regularity. All Criminal and
The real-time video captured from the CCTV camera which Weapons Dataset must be annotated with labels, object
is converted into frames, then the face is detected in the frame bounding boxes.
using the haarcascade algorithm, it is preprocessed. The feature
of the processed real-time image is compared with the feature
of criminal images which are stored in the database. If a
criminal is matched, then the alert is sent to the police personal
which all the details and the time for which he was caught under
the surveillance of camera [1].

Manakula Vinayagar Institute of Technology

Puducherry,
Authorized India
licensed use limited to: Vignana Bharathi Institute Of Technology. Downloaded on July 13,2024 at 07:12:49 UTC fromIEEE ICSCAN
IEEE Xplore. 2021apply.
Restrictions
 ISBN 978-1-6654-3986-2

2. 0.5811631944444444 0.41898148148148145
0.0998263888888889 0.1527777777777778

The values of weapons of Labelled Image in Figure 4

1. 0.42916666666666664 0.38359375
0.425 0.5453125

And we store all the images with corresponding text files in the
same directory for training. Once the dataset is ready, We can
Fig. 2. Labelling Criminal Face 1 start our Testing on Google Colaboratory.

B. Training (Google Colaboratory)

Once done with the dataset, Google Colaboratory is

used for building the YOLOv4 algorithm. The Criminal,
Weapon Images and their respective text files need to be inside
a zip archive. The Zip Archive has to be mounted with Google
Drive. Enable NVIDIA GPU and authenticate Google drive in
google Colaboratory. Also, clone and configure the Darknet on
Google Colaboratory. Darknet is a Framework to train neural
networks for training yolo. Object detection framework for
every image in the dataset is run using Python Script. Then start
the Training. Training must be done for at least 2000 iteration,
Fig. 3. Labelling Criminal Face 2
the more iteration more the model efficiency will be improved
and the reduction also reduced.

Fig. 4. Labelling Weapons

` After opening the ybat annotation tool choose all the criminal Fig. 5. Training the Dataset of Criminal and Weapons
images, then the class file where are the criminal names should
be entered, then click the respective criminal name in class then Maximum iteration makes the system perform with better
annotate all the images. After annotating all the images, click accuracy and efficiency. After all the iterations then we get
the SAVE YOLO button. After saving the YOLO, a text file weights which are trained datasets for our respective criminal's
will be generated for every criminal image which contains the data. Once the data is trained, our system is ready to have Real-
coordinates of bounding boxes like XMin, XMax, Ymin, time data into it to run detections.
YMax. This same procedure is done for labeling the weapons
also.
C. Testing
There are the values of the bounded box of two criminal
shown in Figure 2 and 3. The testing process is used to validate the algorithm. The
data is split in the ratio such that 80% is for training and 20% is
1. 0.6332465277777778 0.24247685185185186 for testing. This trained model is tested with some unseen data
0.3151041666666667 0.3645833333333333 and the model performance is checked. Our model runs at 45

Manakula Vinayagar Institute of Technology

Puducherry,
Authorized India
licensed use limited to: Vignana Bharathi Institute Of Technology. Downloaded on July 13,2024 at 07:12:49 UTC fromIEEE ICSCAN
IEEE Xplore. 2021apply.
Restrictions
 ISBN 978-1-6654-3986-2

frames per second. The dataset we used for training and testing Mean Average Precision: YOLO manages to achieve more
consisted of 100 images per Criminal, the dataset was divided than twice the mean average precision (mAP) of other real-time
80:20 between training and test images for 20000 iterations. systems. Mean Average precision is the average of the average
precision taken for each class. In other words, mean average
precision is the average precision taken over all classes.
D. Detection

After Training and Testing, the system runs on sample

inputs to check whether if there is an error in identifying
criminals. The time taken to detect and the confidence in
identifying the criminals will be generated for every detection
made by the neural network, it can identify all the criminals
present in the image in just 89.6 milliseconds even though the
criminal is partially covered. This model still detects the VII. RESULT AND DISCUSSION
presence of criminals even though it is less confident.
This paper, proposed the criminal face and weapons
detection in real time. We compare the proposed model using
VI. PERFORMANCE MEASURE YOLO with existing Fast R-CNN with respective Precision,
In this paper, the performance of the object and Recall and Mean Average Precision.
criminal detection is measured by Mean Average Precision. TABLE. 1.COMPARISON OF THE RESULTS - YOLO Vs FAST RCNN

Precision and Recall: Precision which is used to measure the Model Precision Recall mAP
correctness of the prediction, and we use Recall to calculate true Fast RCNN 92.56% 89.33% 62.4%
predictions out of total predictions. YOLO 98.56% 91.21% 78.3%

The ratio between the number of correct positive Table 1 shows the comparison of our proposed work
samples to the total number of positive samples, then it is called YOLO and existing Fast RCNN. From the results, it shows that
precision. The model’s accuracy is calculated by the precision YOLO is very effective in running the detection of criminals
classifying positive samples. and weapons. YOLO takes 44 ms to process one image, it
performs the fastest. The Mean Average Precision of proposed
work can achieve 78.3% on testing set while fast RCNN is just
62.4%. The Final Mean Average Precision of our proposed
work is about 78.3% in our testing set, which is best compared
with other object detection algorithm.

TP = True Positives (Predicted as positive as was correct)

VIII. CONCLUSION
FP = False Positives (Predicted as positive but was incorrect)
In this paper, we can monitor the criminals in a real-time
The ratio between the number of correct positive
video stream and find the weapons that are not necessary for
samples to the total number of positive samples, which is used
public places and alert the nearby police cybercrime admin,
for detecting the model’s ability to identify the positive sample.
which increases the security of society and decrease the crime
If the recall is higher than the model detects more positive
rate with the help of technology. We proposed a Real-time
samples.
Criminal detection system using the YOLO algorithm with a
mean average precision of 78.3% and the final average loss of
0.6, which can successfully detect criminals even in crowded
places.

REFERENCES
TP = True Positives (Predicted as positive as was correct)
FN = False Negatives (Failed to predict an object that was there) [1] Apporva.P, Impana. H.C, Siri.S.L, Varshitha.M.R, “Automated Criminal
Identification By Face Recognition Using Open Computer Vision Classifiers”,
International Conference on Computing Methodologies and
Average Precision: To calculate Mean Average Precision, First Communication(ICCMC 2019).
we have to calculate the average precision of every class.
[2] Piyush Chhoriya, “Automated Criminal Identification System using Face
The Average Precision is a single value that represents the Detection and Recognition”, International Research Journal of Engineering and
Technology, October 2019.
average of all precision.

Manakula Vinayagar Institute of Technology

Puducherry,
Authorized India
licensed use limited to: Vignana Bharathi Institute Of Technology. Downloaded on July 13,2024 at 07:12:49 UTC fromIEEE ICSCAN
IEEE Xplore. 2021apply.
Restrictions
 ISBN 978-1-6654-3986-2

[3] Pratibha, Akanksha Gahalot, Uprant, Suraina Dhiman, Lokesh Chouhan,

“Crime Prediction and Analysis”, Institute of Electrical and Electronics
Engineers, Stansford University, 2017.

[4] Justin Lai, Sydney Maples, “Developing a Real-Time Gun Detection

Classifier”.

[5] kruti Goyal, kartikey Agarwal, Rishi Kumar, “Face Detection and
Tracking”, International Conference on Electronics, Communication and
Aerospace Technology(IFECA 2017).

[6] Jazeem Azeez, D. John Aravindhar ,”Hybrid Approach to Crime Prediction

using Deep learning”, International Conference on Advances in Computing,
Communications and Informatics (ICACCIL),2015.

[7] Nurul Iman Hassan, Nooritawati Md. Tahir, Habibah Hashim, “People
Detection System using YOLOv3 Algorithm”, International Conference on
Control System, Computing and Engineering (ICCSCE2020), August 2020.

[8] Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi,”You Only
Look Once: Unified, Real-Time Object Detection”, IEEE Conference on
Computer Vision and Pattern Recognition,2016.

[9] Weijun Chen · Hongbo Huang1,· Shuai Peng1· Changsheng Zhou· Cuiping
Zhang,” YOLO-face: a real-time face detector”, Springer-Verlag GmbH
Germany, part of Springer Nature 2020.

[10] A.S. Osan , V. Thutheja , M. Mulchandani,”AI Based Crime Detection

System”,IOSR Journal of Engineering, May 2019.

[11] Chulyeon Kim, Liyoung Lee, Taekjin Han and Young-Min kim, “A hybrid
framework combining background subtraction and deep neural networks
for rapid person detection”, Journal of Big Data, 2018.

[12] Tanvir Ahmad, Yinglong Ma, Muhammad Yahya, Belal Ahmad, Shah
Nazir and Amin ul Haq, “Object Detection through Modified YOLO Neural
Network”,Hindawi Scientific Programming Volume 2020.

Manakula Vinayagar Institute of Technology

Puducherry,
Authorized India
licensed use limited to: Vignana Bharathi Institute Of Technology. Downloaded on July 13,2024 at 07:12:49 UTC fromIEEE ICSCAN
IEEE Xplore. 2021apply.
Restrictions