IoT based Border Security System using Machine

Learning
Neda Fatima1, Salman Ahmad Siddiqui2, Dr. Anwar Ahmad3
1,2,3Department of Electronics and Communication Engineering, Jamia Millia Islamia, New Delhi, India
E-mail: [email protected] , [email protected], [email protected]
2021 International Conference on Communication, Control and Information Sciences (ICCISc) | 978-1-6654-0295-8/21/$31.00 ©2021 IEEE | DOI: 10.1109/ICCISc52257.2021.9484934

Abstract— Globalization and movement of global citizens barriers in the form of CIBMS in multiple phases.[2]
have rendered the concept of physical barriers between nations Comprising of microwave communication, optical fibre
obsolete. Heavily armed and artillery based physical borders cables, cameras, and an intrusion detection device, this
require a lot of man power, are prone to human errors and system is called BOLD-QIT[1] (Border Electronically
may even harm the environment in the rugged terrains. The Dominated QRT Interception Technique) and was
present paper attempts to demonstrate a novel alternative to established under the Comprehensive Integrated Border
the physical barriers in terms of a ‘Smart Border’ with an Management System.
inbuilt Intruder Alert System that replaces the physical and
armed patrolling with advanced surveillance technology. This Keeping the above project in mind, the paper presents a
system could solve the issues of border security without need comprehensive IoT based intruder alert system that has been
for physical barriers and heavily armed patrolling that takes a developed that uses Machine Learning to detect intruders
toll on human lives and technological resources. The developed along the border with sensors and camera and alerts the
system employs sensors along with surveillance cameras and nearby patrolling station about a possible intruder.
thermal imaging to detect intruders, differentiates intruders Furthermore, the captured data (image) is encrypted and has
with patrol officers and soldiers through machine learning and been stored on cloud to access in future and to avert a
alert officials in case of emergencies. possible security breach.
Keywords— Arduino, Geo tagging, Intruder Alert, IoT,
Machine learning, Smart border II. LITERATURE REVIEW
The section describes the research works in this arena to
I. INTRODUCTION understand in depth the advantages of Smart border and
Globalization is the process by which the world is becoming border surveillance and management using technology.
increasingly interconnected as a result of massively increased
trade and cultural exchange. Globalization has increased the The paper [3] presents a survey of different kinds of
production and movement of goods and services across the intrusion detection systems in smart environments and lists
world. This movement of goods and people has rendered the its advantages as well as anomalies. In [4], the paper
concept of physical barriers between nation states obsolete. theoretically demonstrates an Infra red sensor based border
The concepts of ‘Global Village’ and ‘Global Citizen’ have security system that has the limitation of reporting false
developed much more nuanced meanings in the current positives. In [5], a system has been proposed with the help
scenario. Thus, the concept of Smart Border or Smart Wall of GPS and satellite transmission for automatic activation of
instead of physical borders has gained currency. weapon system in occurrence of warfare. Furthermore, in
India too, shares a long and extensive border of 15,000 km [6], methods have been demonstrated to build an implanted
with seven countries comprising of varied and extreme intruder identification framework in border by utilizing IR
climate and topography. Multiple armed forces patrol these sensor using IOT. The paper [7] proposes a robot which
regions of rugged topography and severe weather conditions identifies terrorist using IR sensor and capture image of
throughout the year. Some even lose their lives due these terrorist using pi camera and sends notification to respective
severe conditions. An alternative to the use of such heavily admin. In [8], the paper presents the concept of the uses
patrolled areas could be to harmonize human resources with thermal imaging camera (FLIR) for detection of various
technology. With this vision in mind, Comprehensive objects and infiltrators. The paper [9] has developed a
Integrated Border Management system (CIBMS) has been human thermal infrared information acquisition platform
envisioned by the Ministry of Defense in Government of that accumulates pyroelectric infrared information to
India. accurately identify human targets. In [10], the author
CIBMS is a five-layer security system for border proposes a network system and an algorithm for a rescue
management to implement the D4R2 (deter, detect, robot to obtain its position under collapsed area. The paper
discriminate, delay, response, recover) principle on the [11] describes home security system using Raspberry Pi and
border. The CIBMS intends to use low-light CCTV cameras, IoT. Finally in [12], the authors demonstrate the most
thermal imaging, night-vision devices (NVDs), surveillance suitable path to carry out rescue operations by using path
radars, laser beams and underground monitoring sensors to learning algorithms.
detect infiltration via land, underwater, air and tunnels. It
includes the integration of manpower, sensors, networks, The present paper addresses the shortcomings of the
intelligence and command & control solutions to improve previous research like errors in intruder detection, reports of
situational awareness at different levels of the hierarchy in false positives, inefficient machine learning model, incorrect
the border guarding forces to facilitate prompt and informed intrusion detection (if uniformed intruder) and inefficient
decision making and quick response to emerging situations. memory utilization and data security issues.
The Indian Government has taken an initiative to cover the
area where physical fence is not feasible with non-physical

978-1-6654-0295-8/21/$31.00 ©2021 IEEE

Authorized licensed use limited to: University of Gothenburg. Downloaded on September 01,2021 at 07:12:59 UTC from IEEE Xplore. Restrictions apply.
It attempts to present a comprehensive solution goes on to inform of possible security intrusion. If a soldier
implementing an IoT based Border Surveillance system that with matching face data is detected, the system continues its
uses data from Raspberry Pi Camera that allows for monitoring.
continuous monitoring and detection as alerts are sent as
soon as an intruder is detected. It is a novel approach
towards border surveillance that uses cloud memory to store
data after encrypting it and access border locations in real
time. This limits the memory redundancy and wastage that
is usually the issue with IoT and its applications.
Furthermore, it utilizes two machine learning algorithms to
accurately detect the soldier; first, it detects whether the
individual is in uniform or not and then following soldier
uniform detection, face detection algorithm is used to detect
intruder in case he is wearing army uniform. The machine
learning algorithms are trained using a more efficient
faster_rcnn_inception_v2_coco model (better loss
parameters and speed than EfficientDet D0 model) to list
individuals detected as soldier or intruder.
III. METHODOLOGY
The methodology and functioning of the system is described
through block illustration representing the system
architecture followed by the system overview to demonstrate
its functioning. Detailed methodology has been described in
step wise manner. Finally, it demonstrates the schematic
built up and the consequent system hardware that has been
built for system implementation.
Fig. 2. System Overview
A. Block Illustration
The block diagram illustrates the fabrication and C. Detailed Methodology
construction of the system. The detailed methodology explains the Intruder Alert
System followed by steps in Training of Models in machine
learning algorithms to detect the intruder.
1) Intruder Alert System
1. A NoIR Pi Camera using Raspberry Pi has been
employed in this system. Raspberry Pi NoIR Camera
V2 Module is 8 megapixel Sony IMX219 Image
sensor custom made for Raspberry Pi Kits. The ‘No
Infra Red’ filter in the camera lens makes it suitable
for night time and low light photography. It can
process 3280 x 2464 pixel static images and also
supports 1080p30, 720p60 and 640x480p60/90
video. This camera is installed to detect a moving
individual/object at the location.
2. The camera feeds the data (video stream) into the
trained machine learning model for object detection
that is stored at the server of the army base station.
Fig. 1. Block Illustration of the technique
3. The Video can also be monitored in real time for
B. System Overview periodic checks. Raspberry Pi is used as an
The flowchart in Figure 2 gives an overview of the intermediate to monitor real time scenario and sends
deployment and functioning of the system. The system has the video stream for the trained model saved on a PC
been set up with a Pi Camera connected to a Raspberry Pi. (server) to detect possible intrusion.
For the proposed algorithm, the server located at Base 4. The GSM Module SIM 800L connected to Arduino
board immediately sends an SMS to the listed number
Station will fetch the video stream from the video server
regarding an individual (intruder) detection or email
running on Raspberry Pi. If an individual is detected in the
alerts can also be sent.
video frame, two scenarios are tested. Firstly, if the 5. In case of intruder detection, the piezo buzzer goes
individual is detected in uniform, then another face ON to intimate about a possible attack or organized
recognition algorithm is activated to check whether the intrusion.
individual detected in soldier uniform is actually a deployed 6. When intruder is detected, the photo along with the
army officer. In both cases, i.e. when a non uniformed timings of detection is made to be saved on cloud. To
personnel is detected and when a uniformed personnel but this effect, Google Firebase cloud service has been
with no matching face data is detected, the buzzer/alarm used.

Authorized licensed use limited to: University of Gothenburg. Downloaded on September 01,2021 at 07:12:59 UTC from IEEE Xplore. Restrictions apply.
7. The photo captured is encrypted before saving on model as it provided superior results as shown in
cloud using AES encryption technique with each Table 1.
photo having a unique cryptographically secured key. 13. Inference Graph is exported from the trained model
This has been demonstrated in Figure 15. that can be used to run the model.
8. The key of each encrypted photo is also saved along
with it on cloud for the purpose of decryption. D. Model Training Flowchart
9. Access to Cloud account is provided to only the The flow chart in Figure 3 represents the training model
designated officials who can authenticate the photos methodology as implemented in the present system. The
in future. same sequence has been employed for EfficientDet D0
2) Training of Model 512*512 model.
Two models faster_rcnn_inception_v2_coco model and
EfficientDet D0 512*512 model have been tested to
determine which provides better parameters. The steps
followed are same in both the models.
After setting up the environment, the model is trained in
the following steps:
1. All the employed machine learning algorithms are
trained on a PC using TensorFlow object detection
API.
2. Tensorflow provides a wide range of models for
pre-trained dataset models such as the COCO
dataset, the Kitti Dataset, the Open Image Dataset,
and the iNaturalist Species Detection Dataset.
Among the COCO trained models,
faster_rcnn_inception_v2_COCO has been utilized.
3. Customized datasets have been built for Uniformed
Individual detection using images of soldier,
soldiers in uniform, intruders and intruders in
different dresses from Google images and in Real
time. Total 300 images were utilized for building
Fig. 3. Model Training Flowchart
the model in which about 200 included soldiers in
uniform and rest were for intruders. E. Schematic Diagram
4. For the second part i.e. face detection of the
uniformed individual, the same set of datasets have
been utilized for detection.
5. As a rule of thumb, dataset has been split for
training and testing purposes in 80:20 ratio. 240
images have been used for training and 60 for
testing.
6. Image is resized to suit appropriate image file
specifications.
7. Image is labeled as Soldier (if uniformed) and
Intruder (if in civilian dress). Fig. 4. Schematic Diagram
8. PASCAL VOC format has been used for labeling
the image and the image generated is saved as an F. System Hardware
XML file.
9. Two CSV files are generated from XML file: Arduino
Testing file and Training file in which the ratios of
images present are in 80:20 in the system. Buzzer
10. With the images labeled, TF Records (binary
storage format) have been created that can serve as GSM Module
input data for training the object detector. TF Pi Camera
record is generated by CSV files that are used to
train the model. Raspberry Pi
11. Then a Label Map file is generated that includes
the number of labeling used and the details of
labeling used.
12. In the next step, training config file is generated Fig. 5. System Hardware
based on the model of choice. The system IV. RESULT ANALYSIS
presented used faster_rcnn_inception_v2_coco
This section presents an extensive analysis of observations,
results and its distinguished features.

Authorized licensed use limited to: University of Gothenburg. Downloaded on September 01,2021 at 07:12:59 UTC from IEEE Xplore. Restrictions apply.
A. Result Analysis 5. The images that have an intruder person or intruder
1) Comparison of faster_rcnn_inception_v2_coco model in army dress are saved on the cloud. The image are
saved after extracting and encrypting with unique
with EfficientDet D0 model
cryptographically secured key for each image using
1. First model was trained using EfficientDet D0 AES encryption along with the timing and location
512*512 model using the steps mentioned in thus efficiently utilizing the cloud storage space.
Figure 2. 6. From the Confusion Matrix generated as shown in
2. Total 2800 steps were initialized for training and Figure 6, precision and recall have been calculated
when it was found that loss parameters were not at IoU (intersection over union)=>0.5 viz.
reducing training was stopped. Thereafter, detections with score 0.5 (threshold) or greater are
inference graph was generated which helps to considered.
create the working procedure of the model. Figure 3) Face Detection Algorithm Results
14 demonstrates the loss and accuracy for the 1. To avoid an event in which an imposter is wearing
model trained using EfficientDet D0 512*512. the uniform for deceiving the system, face detection
3. The maximum loss was above 3 as can be seen algorithm has been employed using
from Figure 13 and the minimum loss was around faster_rcnn_inception_v2_coco model which will
0.9. recognize faces of the soldiers posted at that base
4. During the training, it periodically saved a station along with the uniform as shown in Figures 9
checkpoint every five minutes. This checkpoint has and 10.
been used to export inference graph (.pb file) that 2. Total 1600 steps were initialized with maximum loss
contains graph variables frozen as constants. of 1.3 and minimum loss of 0.0317 as depicted in
Training took around 3 hours using NVIDIA Figure 15.
GEForce GT 710. 3. The face detection algorithm works accurately as it
detects face of soldier to correctly identify in image
5. The second training was performed using
of both soldier (Soldier X) and intruder as depicted
faster_rcnn_inceptionV2 model with a predefined
in Figure 9a and 9b.
initial learning rate of 0.0002 in the model file and 4. Figure 9c depicts the accurate stepwise
decaying after define number of steps. However, implementation of face detection algorithm after the
for this case, the training steps were within the intruder detection algorithm.
limits of initial learning rate steps so it won’t decay 5. Figures 9a and 9b is the result of model testing in
as shown in Figure 13. real time video stream and 9c is the result based on
6. Total 3200 steps were initialized and after that system overview in Figure 2.
training was stopped as loss was acceptable. 6. It also correctly identifies the soldier (Soldier Y) in a
7. The minimum loss was 0.02 and maximum was 1.7 combination of three individuals in armed uniform
as shown in Figure 12. as depicted in Figure 10a and 10b.
8. Thus, faster_rcnn_inception_v2 model performed
B. Experimental Results
better than EfficientDet D0 512*512 model as this
had better loss parameters. This model took around
2 hours for training. Hence, it is better not just from
both loss perspective but also operates faster. This
justifies its adoption for deployment in both
intruder detection and face detection algorithm.
TABLE I: LOSS PARAMETER COMPARISON FOR TWO MODELS

Model Min loss Max loss Avg loss Fig. 6. Precision and Recall from Confusion Matrix
EfficientDet D0 0.9 3.1 1.714
Faster_Rcnn 0.0219 1.7 0.134

2) Intruder Detection Algorithm Results

1. Faster_rcnn_inception_v2 model testing was
performed on set of images obtained from datasets
and the bounding box perfectly encloses the object
detected along with prediction percentage being
displayed.
2. The result accuracy is more than 90%. The results
show accurate detection for of every 8 out of 10
images. This is depicted in Figures 7a, 7b, 7c, 7d.
3. Thereafter, a live video was tested using the trained
model. The intruder detection in live video was also Fig. 7a and 7b. Detected Output Image using Intruder Detection
working accurately during video streaming as Algorithm
shown in Figure 8a and 8b.
4. With the SMS alerts being sent on time by the GSM
Module to the server, officers can be made aware of
the situation near border.

Authorized licensed use limited to: University of Gothenburg. Downloaded on September 01,2021 at 07:12:59 UTC from IEEE Xplore. Restrictions apply.
 Fig. 7c and 7d. Detected Output Image using Intruder Detection
Algorithm

Fig. 11a. Model Training steps for Faster RCNN model

Fig. 8a and 8b. Detected Output Image from Real Time Video Stream
using Intruder Detection Algorithm

Fig. 11b. Model Training steps for EfficientDet D0 model

Fig. 9a and 9b. Face Detection of Soldier X in Real Time Video Stream
using Face Detection Algorithm

Fig. 12. Total loss versus number of steps for Intruder Detection
using Faster RCNN

Fig. 9c. Intruder and Face Detection

Fig. 10a. Face Detection of
in Real Time Video Stream based
Soldier Y using Face Detection
on System Overview
Algorithm
Fig. 13. Model Learning Rate for Faster RCNN

Figure 10b: Face Detection of Soldier Y in a group of Uniformed

Individuals (But not Soldiers) Fig. 14. Total loss versus number of steps using for EfficientDet
D0

Authorized licensed use limited to: University of Gothenburg. Downloaded on September 01,2021 at 07:12:59 UTC from IEEE Xplore. Restrictions apply.
 V. CONCLUSION
The IoT based smart border surveillance system offers a
nuanced vigilance without the need for heavily armed and
militarized borders especially in areas of rugged topography
and extreme climatic conditions thus protecting the lives of
army officials and soldiers. The system represents a classic
example of application of technology for border security in
the countries especially in the current globalised world
without replacing the human and intelligent aspect
associated with it. An efficient, viable and affordable
implementation of the system has been presented with
Fig. 15. Total loss versus number of steps for Face Detection secured access and memory efficiency.
using Faster_RCNN
The system is limited in the case of mask detection. It will
detect a soldier wearing a face mask as an intruder to be on
the safe side. For accurate detection, retina matching
techniques may be applied with high resolution cameras.
Further improvement of the system can be envisioned by
application of more targeted and accurate modern sensors;
however, the cost of the system may escalate in that case.
REFERENCES
Fig. 16. Secured Cloud Storage of Encrypted Images
[1] https://pib.gov.in/PressReleasePage.aspx?PRID=1567263
[2] http://ficci.in/spdocument/23158/smart-Border-Management.pdf
[3] M. F. Elrawy, A. I. Awad, and H. F. A. Hamed, “Intrusion detection
systems for IoT-based smart environments: a survey,” J. Cloud
Comput., vol. 7, no. 1, pp. 1–20, 2018, doi: 10.1186/s13677-018-
0123-6.
[4] R. Karthick, A. M. Prabaharan, and ..., “Internet of things based high
security border surveillance strategy,” … Appl. Sci. …, no. May,
2019,[Online].
[5] M. Of, B. Security, and U. Iot, “Original Research Paper
Engineering,” vol. 2, no. 3, pp. 18–19, 2018.
[6] D. H. M. Ishra, D. E. K. Umar, and V. A. S. Inghal, “Border Security
System using Arduino & Ultrasonic Sensors,” vol. 06, no. 13, pp.
2489–2492, 2017.
[7] K. B. Sri, K. Sachdeva, S. Nikita, and D. C. Umayal, “Border
Security Robot using IoT,” Int. J. Adv. Res. Methodol. Eng. Technol.,
Fig. 17. Proposed Architecture vol. 2, no. 3, pp. 1–6, 2018.
[8] D. A. Lshukri, L. Vidhya Lavanya, E. P. Sumesh, and P. Krishnan,
C. Distinguished Features of the System “Intelligent border security intrusion detection using IoT and
embedded systems,” 2019 4th MEC Int. Conf. Big Data Smart City,
1. The wireless border monitoring system offers a modern ICBDSC 2019, pp. 1–3, 2019, doi: 10.1109/ICBDSC.2019.8645587.
and inexpensive alternative to 24x7 heavily armed and [9] Research on the Multiple Factors Influencing Human Identification
guarded borders especially in rugged areas and extreme Based on Pyroelectric Infrared Sensors, Junwei Yan, Ping Lou, Ruiya
climate conditions. Li, Jianmin Hu, and Ji Xiong, 2018, 604.
2. The system employs a more efficient Machine Learning [10] Miyama, S.; Imai, M.; Anzai, Y.; "Rescue robot under disaster
model after testing of two machine learning models. situation: position acquisition with Omni-directional video footage
sensor detection and robot movement Intelligent Robots and Systems,
3. The system implements efficient memory utilization by IROS, 2003, vol.3, pp. 3132 - 3137.
using cloud storage instead of external memory devices.
[11] Aishwarya D, Dr. J Arokia Renjith, Enhanced Home Security Using
4. Two level Data security implementation using limited IOT and Raspberry, IRJET, 2017, Volume: 04 Issue: 04, 3155-3158.
access and encryption techniques to avoid breach of [12] Chou Wusheng. Wang Tianmiao, You Song, "Sensor-based
security. The data is securely accessed by only autonomous Control for telerobotic system", Proceedings of the 4th
authorized official having access to cloud account and World Congress on Intelligent Control and Automation, 2002, vol.3,
the decryption key. pp. 2430 - 2434.
5. This system is adjustable and can be modified to utilize
modern sensors to improve performance. Furthermore,
enlarged datasets can also be used to improve results.

Authorized licensed use limited to: University of Gothenburg. Downloaded on September 01,2021 at 07:12:59 UTC from IEEE Xplore. Restrictions apply.