International Journal of Computer Science and Information Security (IJCSIS),

Vol. 16, No. 8, August 2018

Deep Learning based Vehicle Detection and Tracking Techniques: State-of-

the-Art Survey
Vikram Kumar Ashima Singh
Research Scholar, Department of Computer Science Assistant Professor, Department of Computer Science
Thapar Institute of Engineering and Technology, Patiala, Thapar Institute of Engineering and Technology, Patiala,
Punjab, India Punjab, India.
Email:[email protected] Email: [email protected]

Abstract——Vehicle detection has become anessential task II. LITERATURE SURVEY

because of the rising usage of surveillance cameras in smart
cities, road network managent, highway and urban traffic Tayara et al. [1] proposed vehicle detection and tallying
planning etc.But detection of vehicles faces many challenges framework for airborne applications.The configuration uses
such as occluded vehicles, shadows of structures, similarity in convolution neural networks to relapse a vehicle spatial
designs of vehicle leading to classification issues. Deep thickness outline the aeronautical picture. A completely
Learning based algorithms such as CNN, RCNN, Faster CNN convolutional relapse network is utilized as a part of this
etc. provides appropriate solution to facilitate vehicle detection framework whose objective is to limit the error between the
because of the self learning capability of the algorithm after ground truth and anticipated yield. It comprises of two
training. This paper aims to present an overview of various ways: up-inspecting way and down-testing way. The down-
vehicle detection techniques based on deep learning which can
examining way is pre-prepared VGG-16 network which has
effectively be used for video surveillance in highway, road and
traffic management systems. rehashed cushioned 3x3 convolutionsfollowed by a
maximum pooling activity. They use upto layer conv5 from
Keywords—CNN, RCNN, Faster CNN, Deep Learning, VGG-16 networkandrest of layers are removed to reduce the
Vehicle Detection, Occlusion. number of parameters. The up-sampling path symmetry is
removed due to the assymetric nature of image regression.
I. INTRODUCTION A 2D circular Gaussian function is used for creating the
Vision-Based Intelligent Transportation System (ITS) is one ground-truth from the dataset. So as to expand the quantity
of the critical applications of video-based supervision of preparing illustrations, information expansion methods,
frameworks. To disentanglevaluable and exact vehicle for example, revolution, horizontall and vertical flipping and
movement data, some of the most researched topics are moving are utilized. At last, a basic associated segment
transportation management and their activity building calculation is utilized to find the areas and count of the
applications. The data for examination of activity in picture blobs. The proposed framework is assessed on two open
and movement stream control data like vehicle direction, datasets to be specific DLR Munich Vehicle dataset gave by
vehicle following, vehicle stream, vehicle order, movement the German Aerospace Center and Overhead Imagery
thickness, vehicle speed, movement path changes, and so Research Data Set (OIRDS) dataset. The proposed
forth. Previously, automation of toll-collect framework used framework can recognize the vehicles in flying pictures
vehicle recognition, division to decide the charge for precisely with better accuracy scores. The research also
different kind of vehicles. As of late, vehicle devours additional time amid derivation contrasted with
acknowledgment framework is utilized for: 1 identification different frameworks.
of the vehicles; 2 detection of the activity paths; 3 ordering
the kind of vehicle class on expressways and streets. The Chu et al.proposeda novel vehicle detection scheme in view
customary vehicle frameworks witnessed great decrease and of multi-assignment profound Convolutional Neural
not perceived well as the vehicles usually are blocked by Networks (CNNs) and Region of Interest (ROI) voting. In
different vehicles or by foundation impediments like street the design, the administered data is enhanced with
signals, trees, and climate conditions. Moreover, the subcategory, jumping box relapse, region cover, and class of
executions of these frameworks rely upon a decent activity- each training ROI as a mutli-assignment learning-based
picture examination ways to deal with recognization, framework.This configuration enhances the adequacy of
tracking and characterization of the vehicles. This review powerful detection by permitting the CNN model to share
paper explores the various deep learning based vehicle visual information among various vehicle traits at the same
detection techniques used in applications such as traffic time. This outline uses the CNN model to anticipate the
surveillance, battleground survey, autonomous vehicles and offset course of every rous limit towards the relating ground
aerial survey. truth. A subcategory NMS strategy is acquainted with

53 https://sites.google.com/site/ijcsis/
ISSN 1947-5500
 International Journal of Computer Science and Information Security (IJCSIS),
Vol. 16, No. 8, August 2018

conquered the difficulties confronted while having impeded background to the classification. CNN model was inputed
vehicles. At that point every rous can vote in favor of the focused image for classification. Reinforced CNN is
appropriate contiguous bouncing boxes that are reliable with then implemented using information entropy calculated
the extra data. The scores are contrasted and the scores of based on classification probability distribution. This helped
the individual ROI itself to acquire more exact outcomes. the learning agent to achieve more accurate image
Test comes about are discovered utilizing KITTI PC vision classification. This is done by an evaluation network. After
benchmarks and the PASCAL 2007 vehicle dataset is obtaining the important areas, variousinput components
utilized for preparing. The outcomes showssuperior such as vehicle lights, vehicle wiper, etc are extracted from
execution to existing distributed works. the imge and these feaures are used to detect and locate
vehicles. Systemataic experiments on a surveillance-nature
Cai et al. [3] suggested a versatile scene vehicle detection datasets which contains images captured by surveillance
calculation in view of composite profound structure. This cameras in the front view are conducted in form of two
outline is propelled by the Bagging (Bootstrap Aggregating) tasks. The first task ‘Vehicle-5’ is to divide vehicles into 5
instrument. Various generally autonomous source tests are caegories according to their types. The second task
first used to fabricate different classifiers and after that ‘Vehicle-58’ is vehicle classification discriminated from
voting is utilized to create target preparing tests with different vehicle markers. The design requires all images to
certainty scores. The programmed highlight extraction be sqaures andhence all the images are compressed into
capacity of profound convolutional neural networks is then squares 224x224. The model proved to be more vesaltile
used to perform source-target scene include comparability than large-scale CNN.
clcultions with a depp auto-encoder keeping in mind the end
goal to outline a composite profound structure based scene- Xiaozhu et al.[6] implemented a novel mehodto detect
versatile classifier and its preparation technique. Tests on armored vehicles in a battlefield. The background becomes
the KITTI dataset and a dataset caught by the gathering complex in a battlefield due to artificial camoflague, varying
exhibit that the proposed strategy performs superior to illumination, shelters, etc. This paper detects armored
anything existing machine-learning based vehicle detection vehicles using deep learning. The genuine combat zone
strategies. What's more, contrasted and existing scene- condition picture is first made into the VOC2007, and the
adpative protest detection techniques, the proposed strategy information is prepared by the Faster R-CNN and ZF NET
enhances the detection rate by a normal of roughly 3%. model. The proposed demonstrate utilizes a particular
pursuit strategy contrasted with the customary sliding
Deng et al. [4 ]explored Region-based convolutional neural window system of the traditional R-CNN technique. CNN is
networks (R-CNNs) have a few difficulties when utilized utilized for highlight extraction and standard database
for vehicle detection in ethereal pictures. Right off the bat, (ImagNet, Pascal VOC) is utilized for preparing and
the vehicles are moderately little in measure in vast scale assessment. As indicated by the test results,it is
elevated pictures. Also, R-CNNs are intended to identify the demonstrated that the Faster RCNN ZF display goodly
jumping box of the objectives without extractin the affects the detection and acknowledgment of shielded
properties of the question. Thirdly, manual explanation is defensively covered vehicles in front line condition.
costly and the accessible manual comment of vehicles for Contrasted and the conventional method,the question
preparing R-CNNs are lacking in nature. To address these detection and acknowledgment technique in view of the
issues, a quick and precise veicle detection system is profound learning kills the bulky picture preprocessing
proposed. On one hand to precisely extricate vehicle-like link,and its conclusion to-end engineering enormously
targets, theydeveloped an exact vehicle-proposition network enhances the detection and acknowledgment effectiveness,
(AVPN) in light of hyperfeature outline joins hierarchial and has great versatility,showing a solid application
include maps that are more precise for shopping center objet prospect. Furthur inquire about proposed is to build the
detection. On other hand, they propose a coupled R-CNN acknowledgment precision and the exactness of the
strategy, which joins an AVPN and avehicle property bouonding box relapse.
learning netork toextract the vehicle's area and qualities at
the same time. For unique extensive scale airborne pictures Wang et al. [7] found Vehicle-following maps areimportant
with restricted manual explanations, they utilized trimmed part of autonomous driving frameworks. Locating and
picture blokcs for traiing with information expansion to acknowledging presence of tail- light is vital to keep an
abstain from overfitting. Complete investigation on the independent vehicle from backside impacts. Aoustic sonar
general population Munich Vehicle dataset and a gathered Advanced Driving Assistance System (ADAS) are available
dataset show the precision and adequacy of the proposed in market for the purpose. Mobile-eyesare also utilized for
technique. backside crash cautioning. Researchers have built up a novel
approach to deal detecting vehicles and perceiving brake-
Zhao et al. [5] were inspired by the mechanism of the lights from a solitary picture progressively. Not at all like
human vision system and its capability to distinguish past methodologies where match taillight must be removed
nuances processing. It proposes a convolutional neural unequivocally, have researchers utilized vehicle raise
network (CNN) targeting visual attention classifying appearance picture. On an extensive database, "Brake Lights
images.For generating a focused image, highlighting Patterns" (BLP) are learned by a multi-layer observation
selected segment of the image while ignoring other details neural system. Given a picture, the vehicles can be named
was achieved by visual-attention based image processing "brake" or "typical" utilizing the profound classifier. The
module. This helpsto reduce the interferences of the vehicle can be identified rapidly and vigorously by

54 https://sites.google.com/site/ijcsis/
ISSN 1947-5500
 International Journal of Computer Science and Information Security (IJCSIS),
Vol. 16, No. 8, August 2018

consolidating multi-layer lidar (IBEO Lux combination which abuse a two-branch Multi-DNN Fusion Siamese
framework) and a camera. Street division and a novel Neural Network (MFSNN) to wires the order yields of
vanishing point locale of intrigue (ROI) assurance strategy shading, display and stuck blemishes on the windshield and
are investigated to additionally accelerate the identification guide them into an Euclidean space where separation can be
and enhance the strength. The test comes about directed on straightforwardly used to gauge the similitude of
some genuine on-street recordings have demonstrated the discretionary two vehicles. With a specific end goal to
strength and effectiveness of the proposed approach. accomplish this objective, researchers introduce a technique
for vehicle shading recognizable proof in view of Alex net,
Cui et al. [8] Vehicle re-identification has turned into a a strategy for vehicle demonstrate distinguishing proof in
basic undertaking in light of the developing blast in the view of VGG net, a technique for glued imprints recognition
utilization of observation cameras out in the open security. and ID in light of Faster R-CNN. Researchers assess our
The most generally utilized arrangement depends on tag MFSNN technique on VehicleID dataset and in the analysis.
confirmation. Be that as it may, when confronting the Test comes about demonstrate that our technique can
vehicle without a permit, deck autos and other tag data accomplish promising outcome. Table 1 presents a
mistake or missing circumstance, vehicle seeking is as yet a comparison of various vehicle detection and tracking
testing issue. This paper proposed a vehicle re- techniques available in literature.
distinguishing proof technique in light of profound realizing

TABLE 1: COMPARISON OF VARIOUS VEHICLE DETECTION TECHNIQUES BASED ON LITERATURE

Citation Technique used Parameters Solution and
Accuracy
Tayara et al. [1] CNN based model is used to Higher precision and F1 scores compared to Precision:93.30%
regress the vehicle spatial conventional technique but requires more Recall:90.51%
density from an aerial time.
image.
Chu et al. [2] Multi-tasking deep CNN More accurate results on the data tested with Accuracy:91.67%
model alongside ROI voting the KITTI dataset.

Cai et al.[3] Based on the Bootstrap The proposed method improves the detection Detection rate above
Aggregating rate by an average of approximately 3%. 92%
mechanism.Uses deep
composite structures for
detection.
Deng et al.[4] AVPN based on hyper Evaluationon Munich Datasets shows faster Precision:92%
feature map and a coupled and moreefficient performance compared to
R-CNN model existing algorithms.
Zhao et al., [5] Uses a CNN model that The proposed model is more competitive 97.93% test accuracy
focuses on particular section compared to large scale CNN models rate on vehicle-5
of image to identify dataset.
important sections
Xiaozhu et al.6] Faster RCNN and ZF NET eliminates the cumbersome image Detection rate of tank
are coupled to detect preprocessing link, and it’s end-to-end is 97.20% and for
armored vehicles; architecture greatly improves the detection wheeled infantry
and recognition efficiency fighting vehicle
detection rate is
95.70%
Wang et al., [7] BLPs are learned by CNN, The average recognition accuracy is 89% and Vehicle-following,
The vehicle is detected by when applied to the vehicle detection, and the obstacle avoidance,
combining multi-layer lidar average accuracy is 99% etc., have shown that
and a camera. ROI is used the accuracy and
for road segmentation speed of the proposed
approaches are
satisfied with the
navigation
requirement.
Cui et al. [8] A two branch Multi DNN Compared with existed methods, the proposed Accuracy 70.5%
Fusion Siamese Neural network structure achieves a high predict
Network (MFSNN) is used. accuracy
It is based on VGG net and

55 https://sites.google.com/site/ijcsis/
ISSN 1947-5500
 International Journal of Computer Science and Information Security (IJCSIS),
Vol. 16, No. 8, August 2018

faster RCNN.
A selective search method The proposed method is superior to the result Detection rate:86.2%
and a target detection model of multi - feature and classifier detection in
Shi et al. [9]
based on Fast R-CNN are terms of accuracy.
used to detect vehicle
Bedruz (10) minimize the effect of 81% detection rate in
occlusion in real time 83 correct prediction out of 102 high density area
vehicle detection
Saini (11) traffic surveillance for It is able to track vehicle even when there are Maximum number of
accident detection several other vehicles in the camera’s field of feature detection in
view. moving vehicle
Sajib (12) Increase the speed of average detection rate is 97.81% and average Accuracy:97.81%
classification and detection classification accuracy is
91.75%

Arya (13) Detect the vehicles using Efficiency:85-90%

videos of both low and high
resolution captured by static
camera
GU (14) Classification and detection The mAP of our network upto 80.5% in Detection speed is 48
of vehicle using CNN detecting vehicle class frame per second
which can be used for
real time vehicle
detection
Kim(15) New licence plate detection Precision: 98.39% Precision:98.39%
method Recall: 96.84% Recall:96.84%
Zhuang (16) Rapid vehicle detection True positive rate: 96.30 Precision:97.2%
using a cascade of strong Recall:95%
classifiers

Shi et al. [9]As of late the exploration of vehicle location is and power.
fundamentally through machine adapting, yet regardless it Remaining of the vehicle detection techniques using deep
has low identification exactness issue. With the learning approaches can be summarized by following
investigation of analysts, utilizing profound learning studies. Bedruz [10] presented an approach to limit the
strategies for vehicle identification ends up hot. In this impact of impediment on managing constant vehicle
paper, a particular hunt technique and an objective detection and following. The vehicle detection framework
recognition demonstrate in light of Fast R-CNN are utilized will utilize the detection of vehicles going to theregion of
to recognize vehicle. The system advances the model by interest (ROI). Saini[11] The vehicle detection and
preprocessing the example picture and the new system following strategy proposed in this paper requires moving
structure. Right off the bat, the trial utilizes general society highlights to be distinguished and followed additionally
KITTI informational index and self-gathered BUU-T2Y utilizing region based square coordinating methodology.
informational collection, individually, to train approval and Sajib[12] proposed focusing on ongoing applications. So
test. Also, in view of the first informational index, the trials the execution time must be quicker in both detection and
go ahead through incremental picking up, joining the KITTI arrangement. They are proposing a financially savvy and
dataset with the BUU-T2Y dataset. The test comes about quicker model that can without much of a stretch be
demonstrate that the proposed strategy is better than the executed for ongoing applica-tions. Arya[13] aimed to
consequence of multi - highlight and classifier identification experiment and study continuous vehicle detection and
as far as exactness. To an expansive degree, the proposed following calculations. It would limit the imperatives like
technique tackled the issue of missing vehicle for info picture determination, securing stature, natural
recognition and enhanced the precision of vehicle testing commotion and so forth on the information document and

56 https://sites.google.com/site/ijcsis/
ISSN 1947-5500
 International Journal of Computer Science and Information Security (IJCSIS),
Vol. 16, No. 8, August 2018

still ready to accomplish attractive execution. Further, to Deng, Z., Sun, H., Zhou, S., Zhao, J., & Zou, H. (2017).
Toward fast and accurate vehicle detection in aerial
distinguish the vehicles utilizing recordings of both low and images using coupled region-based convolutional
high determination caught by a stationary activity neural networks. IEEE Journal of Selected Topics in
Applied Earth Observations and Remote Sensing, 10(8),
observation camera. GU et al. [14] proposed a method of 3652-3664.
real-time vehicles detection and tracking using Gu, L., Jia, D., Vicaire, P., Yan, T., Luo, L., Tirumala,
Convolutional Neural Networks. They present network A. & Krogh, B. H. (2005, November). Lightweight
detection and classification for wireless sensor
architecture, which create multiple vehicle candidates and networks in realistic environments. In Proceedings of
predict vehicleprobabilities in one evaluation. Kim [15] the 3rd international conference on Embedded
networked sensor systems (pp. 205-217). ACM.
firstly detected vehicle regions with the faster R-CNN
Kim, K. I., Jung, K., & Kim, J. H. (2002). Color
algorithm and localise the license plate in each vehicle texture-based object detection: an application to license
region with a convolutional neural network (CNN) plate localization. In Pattern Recognition with Support
Vector Machines (pp. 293-309). Springer, Berlin,
classifier. Heidelberg.
Saini, R., Ahmed, A., Dogra, D. P., & Roy, P. P.
CONCLUSIONS AND FUTURE SCOPE (2017). Surveillance scene segmentation based on
trajectory classification using supervised learning.
In Proceedings of International Conference on
In the last decade, object detection and tracking remained a Computer Vision and Image Processing (pp. 261-271).
problem in research and it has received huge attention across Springer, Singapore.
the globe. In the present study, various approaches to detect Sajib, M. S. R., & Tareeq, S. M. (2017, December). A
and track objects in videos are presented. This paper surveys feature based method for real time vehicle detection and
classification from on-road videos. In Computer and
deep learning based vehicle detection techniques used in Information Technology (ICCIT), 2017 20th
video processing. It presents a comparative study of the International Conference of (pp. 1-11). IEEE.
algorithms used in each technique and presents a Shi, K., Bao, H., & Ma, N. (2017, December). Forward
comprehensive survey on various proposed techniques for Vehicle Detection Based on Incremental Learning and
vehicle detection. Further, this survey gives a better Fast R-CNN. In Computational Intelligence and
Security (CIS), 2017 13th International Conference
understanding of the various techiques and highlights the on (pp. 73-76). IEEE.
issues and solutions to various challenges faced which can Tayara, H., Soo, K. G., & Chong, K. T. (2018). Vehicle
help researchers in choosing an appropriate technique for Detection and Counting in High-Resolution Aerial
vehicle detection based on the requirements and environment Images Using Convolutional Regression Neural
Network. IEEE Access, 6, 2220-2230.
Wang, J. G., Zhou, L., Pan, Y., Lee, S., Song, Z., Han,
REFERENCES B. S., & Saputra, V. B. (2016, June). Appearance-based
brake-lights recognition using deep learning and vehicle
detection. In Intelligent Vehicles Symposium (IV), 2016
Arya, K. V., Tiwari, S., & Behwalc, S. (2016, June). IEEE (pp. 815-820). IEEE.
Real-time vehicle detection and tracking. In Electrical Xiaozhu, X., & Cheng, H. (2017, July). Object
Engineering/Electronics, Computer, detection of armored vehicles based on deep learning in
Telecommunications and Information Technology battlefield environment. In Information Science and
(ECTI-CON), 2016 13th International Conference Control Engineering (ICISCE), 2017 4th International
on (pp. 1-6). IEEE. Conference on (pp. 1568-1570). IEEE.
Bedruz, R. A., Sybingco, E., Quiros, A. R., Uy, A. C., Zhao, D., Chen, Y., & Lv, L. (2016). Deep
Vicerra, R. R., & Dadios, E. (2016, November). Fuzzy reinforcement learning with visual attention for vehicle
logic based vehicular plate character recognition system classification. IEEE Transactions on Cognitive and
using image segmentation and scale-invariant feature Developmental Systems.
transform. In Region 10 Conference (TENCON), 2016 Zhuang, X., Kang, W., & Wu, Q. (2016). Real-time
IEEE (pp. 676-681). IEEE. vehicle detection with foreground-based cascade
Cai, Y., Wang, H., Zheng, Z. & Sun, X. (2017). Scene- classifier. IET Image Processing, 10(4), 289-296.
Adaptive Vehicle Detection Algorithm based on a
Composite Deep Structure. IEEE Access.
Chu, W., Liu, Y., Shen, C., Cai, D., & Hua, X. S.
(2018). Multi-task vehicle detection with region-of-
interest voting. IEEE Transactions on Image
Processing, 27(1), 432-441.
Cui, C., Sang, N., Gao, C., & Zou, L. (2017,
November). Vehicle re-identification by fusing multiple
deep neural networks. In Image Processing Theory,
Tools and Applications (IPTA), 2017 Seventh
International Conference on (pp. 1-6). IEEE.
Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-
Fei, L. (2009, June). Imagenet: A large-scale
hierarchical image database. In Computer Vision and
Pattern Recognition, 2009. CVPR 2009. IEEE
Conference on (pp. 248-255). Ieee.

57 https://sites.google.com/site/ijcsis/
ISSN 1947-5500