Drone Detection and Identification System using

Artificial Intelligence
Dongkyu ’Roy’ Lee, Woong Gyu La, and Hwangnam Kim
School of Electrical Engineering, Korea University, Seoul, Rep. of Korea
Email: {roylee6315, juhgiyo, hnkim}@korea.ac.kr

Abstract—As drone attracts much interest, the drone industry environment with surveillance drones with following scenario.
has opened their market to ordinary people, making drones to During its mission, the surveillance drone records its surround-
be used in daily lives. However, as it got easier for drone to ing on camera. If there were an unknown drone, it will be
be used by more people, safety and security issues have raised
as accidents are much more likely to happen: colliding into found on the video frame by applying object detection, and
people by losing control or invading secured properties. For safety the system will mark it on the frame. The marked frame is
purposes, it is essential for observers and drone to be aware of an then used further. The system, which has learned various drone
approaching drone. In this paper, we introduce a comprehensive model through machine learning, identifies the unknown drone
drone detection system based on machine learning. This system by model. Such system can be applied on other surveillance
is designed to be operable on drones with camera. Based on the
camera images, the system deduces location on image and vendor system such as [2], and complement its ability to find unknown
model of drone based on machine classification. The system is flying objects.
actually built with OpenCV library. We collected drone imagery For image processing, the system uses GPU with CUDA
and information for learning process. The system’s output shows programming [3] on the video frames. On each frames, ob-
about 89 percent accuracy. jection detection algorithm is applied to spot the drone in the
Index Terms—artificial intelligence, classifier, deep learning,
detection, image processing video frames. Specifically, cascade classification is adapted
for object detection. With machine learning based on various
I. I NTRODUCTION drone images, which are given and processed beforehand, the
system learns how to detect drones accurately from frames.
Recently, the interest and demand in drone are higher Then, frames with drone detected are processed once more to
than ever. With this popular demand, new types of drone identify the drones model type.
merchandise has been designed and manufactured, so that The remainder of this paper is organized as follows. First,
civilians can afford to buy them for various purposes (i.e., we introduce researches by others and explain our research
research, leisure, etc.). As for this shift, the commercial drone validation in Section II. We explain our design of the system,
industry keep on growing. and the implementation details in Section III. We present our
Although commercial drones have been massively produced experiment resutls and evaluate the performance of our system
to satisfy the civilians needs, there has been some downsides in Section IV. Finally, we end our paper with a conclusion in
to this. As it became easier to spot drones outdoors, more Section V.
safety issues has been brought up as concerns. These are not
merely about accidents regrading to drones harming individ- II. R ELATED WORK
uals, but include drones invading government restricted areas. In this section, we introduce various researches relevant
Additionally, considering that a coordinated fleet of drones is to our research. The content of this paper can be summa-
capable of more various tasks [1], drones can be a bigger threat rized as a drone detection and identification system based
than people could imagine. As there are more drones out in on image processing and machine learning. Haar Featrue-
public, it became harder to regulate them legally and safely. based Cascade Classification [4] is applied for object detection.
For regulation, it is essential to detect and identify drones up This technique has three key characteristics. One is that by
in the sky. using Integral Image, features used by the detector can be
When it comes to detection, the first thing that comes computed very quickly, making its computation complexity
to mind is a radar. Among various radars, there are radars O(1). Two is the method constructing the classifier by selecting
specially designed to detect small aircrafts that humans cannot a small number important feature using a learning algorithm,
be on board. However, these radars have limits. For one thing, yielding a very efficient classifier. Three is the method for
it cannot distinguish between birds and drones. This calls for combining increasingly more complex classifiers in cascade
an alternative solutions to detect drones accurately. fashion, making it dramatically fast.
In this paper, we propose a comprehensive drone detection Recently, deep learning has been applied in many studies
system based on machine learning as an alternative solution in image processing field. Among the deep neural network
for the aforementioned matter. This system applies object structures, Convolutional Neural Network (CNN) has been
detection to capture the unknown drones as objects in the gaining great interest because it automatically detects the
image, then identify the unknown drones model type by important features during its learning phase. It has already
matching the captured image of the drone to vendor catalog shown performance breakthroughs in image classification [5]
using machine learning. Our system can be applied in an and object detection [6]. The reason why CNN shows great

978-1-5386-5041-7/18/$31.00 ©2018 IEEE 1131 ICTC 2018

Authorized licensed use limited to: Middlesex University. Downloaded on September 04,2020 at 07:12:15 UTC from IEEE Xplore. Restrictions apply.
Fig. 1: Process of Drone Detection and Identification System.

performance in extracting feature is its capability in extracting

abstract features. This allows it process with greater accuracy
and speed [7]. Fig. 2: Drone Detection from Images.
In this paper, drone identification is based on supervised
learning. Supervised learning is a systematically more adjusted
CNN, and has been applied in [8] and [5]. It has been gaining the 2D area on the frame, where the drone is located. Then, the
recognition by showing better performance than unsupervised area is cropped, and passed on to the drone identifier module.
learning [9].
Based on the aforementioned researches, we have chosen C. Drone Identifier Module
CNN to increase speed and efficiency. The two modules imple- The drone identifier is a module that classifies the unknown
mented in this system requires learning based on images. As it drone into vendor models based the cropped image. To identify
heavily relies on machine learning, with careful calibration and the model type of the drone, a CNN is constructed. Drone
sufficient learning, the system can show better performance. detection and drone identification can be done by other types
of deep learning such as ResNet [11] or Raster R-CNN [6].
III. S YSTEM DESIGN AND I MPLEMENTATION However, due to its depth, it would take a very long time
In this section, we describe our proposed system and for training, and the complexity is too great to construct. To
implementation details. We first explain the overall system. overcome this, we have separated the system into a classifier
Then, we describe each modules in detail. and a CNN for processing. Additionally, this allows us to
train faster with fewer image. The CNN consists of two Conv
A. Overall System
layers and two fully connected layers. Each fully connected
The proposed system is designed to be on top of a comput- layer are implemented with a 30 percent dropout [12]. For the
ing board established on each surveillance drones. After going activation function, Rectified Linear Unit (ReLu) [13] is used,
through the process described in Fig.1, the drones are to report and Adaptive Motion Estimation (Adam) [14] is used for the
to the Ground Control Station (GCS). gradient descent algorithm.
B. Drone Detector Module IV. P ERFOMANCE E VALUAITON
The drone detector is module that detects drone on video In this section, we show our experiment results and per-
frames taken by the camera on the drones. Accurate 2D formance analysis. First, we show how the drone detection
detection on the frame is key to the overall system. There- module works. Then, we analyze the performance of the drone
fore, to ensure accuracy while maintaining speed, our system identifier module.
adapts Haar Feature-based Cascade Classifier of OpenCV
library [10]. For training the classifier, we used 2088 pos- A. Drone Detector Experiment
itive examples and 3019 negative examples. We collected Before running the drone detector module, it is required
the positive examples from Google, and manual cropped the to go through a learning phase. For the learning, the images
drone from every images. Negative examples were collected collected from Google has been used for the drone detectors
from http://face.urtho.net/. By applying image distortion on the supervised learning. After using the 2088 positive example
positive examples, we expanded our positive example pool to images, we have prepared 829 test images for testing the per-
7000. Based on these positive images, we trained our Haar formance of the system. These test images were not used for
Feature-based Cascade Classifier. The trained classifier finds the learning process, and have only been used for performance

1132

Authorized licensed use limited to: Middlesex University. Downloaded on September 04,2020 at 07:12:15 UTC from IEEE Xplore. Restrictions apply.
 V. C ONCLUSION
In this paper, we propose a drone detection and identifi-
cation system making decision based on video frames. This
system has shown that even with simple artificial intelligence,
the performance is very promising. All systems were actually
implemented, and training data were collected from the web.
With small amount of easily collectable training data, the
system still showed great accuracy, which makes it more
appealing.
For future work, we would like to develop a distance
estimation module to complement the existing system. Based
images, estimating the distance between the surveillance drone
and the unknown drone can be valuable information, and can
be used for tracking.
ACKNOWLEDGMENT
This research was supported by Unmanned Vehicles
Fig. 3: False Classified Drone Images. Advanced Core Technology Research and Development
Program through the Unmanned Vehicle Advanced Re-
search Center(UVARC) funded by the Ministry of Science,
ICT and Future Planning, the Republic of Korea (NRF-
2016M1B3A1A01937599)
R EFERENCES
[1] S. Yoo, J. Jung, A. Y. Chung, K. Kim, J. Lee, S. Park, S. K. Lee, H. K.
Lee, and H. Kim, “Empowering drones teamwork with airborne net-
work,” in Advanced Information Networking and Applications (AINA),
2017 IEEE 31st International Conference on. IEEE, 2017, pp. 678–685.
[2] J. Jung, S. Yoo, W. La, D. Lee, M. Bae, and H. Kim, “Avss: Airborne
video surveillance system,” Sensors, vol. 18, no. 6, p. 1939, 2018.
[3] S. Cook, CUDA programming: a developer’s guide to parallel comput-
ing with GPUs. Newnes, 2012.
[4] P. Viola and M. Jones, “Rapid object detection using a boosted cascade
of simple features,” in Computer Vision and Pattern Recognition, 2001.
CVPR 2001. Proceedings of the 2001 IEEE Computer Society Confer-
ence on, vol. 1. IEEE, 2001, pp. I–I.
[5] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image
recognition,” in Proceedings of the IEEE conference on computer vision
and pattern recognition, 2016, pp. 770–778.
Fig. 4: Accuracy of Deep CNN along with iteration. [6] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time
object detection with region proposal networks,” in Advances in neural
information processing systems, 2015, pp. 91–99.
[7] W. Shi, J. Caballero, F. Huszár, J. Totz, A. P. Aitken, R. Bishop,
evaluation. This is to validate that our system can still detect D. Rueckert, and Z. Wang, “Real-time single image and video super-
drones from unfamiliar images that has not been exposed. resolution using an efficient sub-pixel convolutional neural network,” in
Fig.2 are the detection results of the drone detector. As Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, 2016, pp. 1874–1883.
shown in the image, if the system estimates that there is a [8] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan,
drone in the image, then the area, where the drone is expected V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,”
to be, is marked in a rectangular box. in Proceedings of the IEEE conference on computer vision and pattern
recognition, 2015, pp. 1–9.
B. Drone Identifier Experiment [9] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521,
no. 7553, p. 436, 2015.
The drone identifier is to identify the drone model. The [10] G. Bradski and A. Kaehler, “Opencv,” Dr. Dobbs journal of software
tools, 2000.
2088 positive example images were all manually labeled. 1777 [11] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, “Inception-v4,
image among the positive example images were used for inception-resnet and the impact of residual connections on learning.” in
training, and 429 were used as evaluation test images. Mini- AAAI, vol. 4, 2017, p. 12.
[12] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhut-
batch with 256 batch size was used for training. dinov, “Dropout: a simple way to prevent neural networks from over-
The drone identifier shows maximum of 91.6 percent accu- fitting,” The Journal of Machine Learning Research, vol. 15, no. 1, pp.
racy. False classification are only made in cases that are quite 1929–1958, 2014.
[13] X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural
challanging even for the human eyes as it is shown in Fig.3. networks,” in Proceedings of the fourteenth international conference on
Fig.4 shows that the system reaches its maximum accuracy artificial intelligence and statistics, 2011, pp. 315–323.
within a few iteration, and saturates around 89 percent. This [14] D. P. Kingma and L. Ba, “Adam: a method for stochastic optimization,”
in International Conference on Learning Representations, 2015.
experiment proves that Deep CNN is very efficient for clas-
sifying drones by model. Additionally, it shows that it doesnt
requires much training data.

1133

Authorized licensed use limited to: Middlesex University. Downloaded on September 04,2020 at 07:12:15 UTC from IEEE Xplore. Restrictions apply.