IAES International Journal of Artificial Intelligence (IJ-AI)

Vol. 10, No. 1, March 2021, pp. 66~73

ISSN: 2252-8938, DOI: 10.11591/ijai.v10.i1.pp66-73 66

Vehicle detection and tracking for traffic management

Mallikarjun Anandhalli1, Vishwanth P. Baligar2, Pavana Baligar3, Pooja Deepsir4, Mithali Iti5
1,3,4,5Department
of Electronics and Communication Engineering, KLS Gogte Institute of Technology, India
2Department
of Computer Science Engineering, KLE Technological University, India
3Department of Computer Science Engineering, SKSVM. Agadi College of Engineering & Technology, India

Article Info ABSTRACT

Article history: The detection of object with respect to vehicle and tracking is the most
needed step in computer research area as there is wide investment being
Received Apr 9, 2020 made form intelligent traffic management. Due to changes in vision or
Revised Dec 21, 2020 scenes, detection and tracking of vehicle under different drastic conditions
Accepted Jan 23, 2021 has become most challenging process because of the illumination, shadows.
To overcome this, we propose a method which uses tensorflow fused with
corner points of the vehicles for detection of vehicle and tracking of an
Keywords: vehicle is formulated again, the location of the object which is detected is
passed to track the detected object, using the tracking algorithm based on
CNN CNN. The proposed algorithm gives result of 90.88% accuracy of detection
Corner points in video sequences under different conditions of climate.
TensorFlow
Tracking
Vehicle detection This is an open access article under the CC BY-SA license.

Corresponding Author:
Mallikarjun Anandhalli
Department of Electronics and Communication Engineering
KLS Gogte Institute of Technology
India
Email: [email protected]

1. INTRODUCTION
Since from few decades many methods were proposed to detect and track object in foreground, but
in one or the other way methods couldn’t give accuracy in the challenge [1]. Main aim of the proposed work
is to find the object position to detect and track it which is used in various fields like military, medical and
video analysis, usually frame consists of background as well as foreground data [2], here foreground image is
represented by ROI feature points, and other points are considered as the background. In the surveillance
system it follows two main procedures those are detection of the moving object and calculation of the
motion. First is detection of the object and it is carried out by background pixels data.
Many researchers have developed different methodologies for object detection in video where many
combinations of methods and differences in protocols exist. Background subtraction is used to segment the
moving object in the video [3].
In reality, the optical flow algorithm is used to overcome this disadvantage but when different
conditions are considered, sometimes it gives false alarm for algorithms. In about all the background
subtraction cases, background data is used by object trackers however it leads to mis-classification. Next to
improve the accuracy of this algorithm, selection of robust classifier is used [4].
In the era of increasing population every individual makes use of an automotive industry i,e; the use
of vehicles has been increased rapidly. This made adverse effect on traffic management system. There are no
novel approaches or algorithms for the detection of vehicles running on the road to find the density of
vehicles. All the algorithms proposed till today detect vehicles only in particular climate conditions most of

Journal homepage: http://ijai.iaescore.com

Int J Artif Intell ISSN: 2252-8938 67

the algorithms doesn’t hold good for all the climate conditions. Hence the proposed algorithm makes use of
tensorflow in order to detect objects. And later the objects are tracked to stabilize algorithm for detection of
vehicles.
To overcome this problem, in this proposed method generalized object detection algorithm based on
tensorflow and object tracking algorithm based on CNN has been used. These methods are found robustly
and tracked the objects in the complicated images and background scenes. This is based on this condition that
the images have strong local two-dimensional structures. The CNN combines three architectural ideas to
guarantee a difference in variance of change, up to certain degree and distortion: shared weights, local
receptive fields and sometimes, temporal and spatial sub-sampling. The main problem lies in tracking of the
object in the field of computer vision. Most of the methods such as, blobs movement [5-10], follow-up
balanced which is a basic model that is considered over the time. The challenges faced by these techniques in
real time situations are enormous because:
− Do not have a model that distinguishes itself from others in the category of interest.
− The model from its original can deviate over the time.
Many methods were proposed by researchers’ community from background substitution to CNN.
Few algorithms have been listed here and discussed. Survey gives objects feature point detection,
segmentation, classification algorithm and background subtraction. To have exact tracking, features of object
are needed therefore detection of object has main role here. Probabilistic and deterministic motion model and
the model based on appearance are adapted to get better accuracy. The process of tracking involves training
of the feature points and updating. Main problem lies when tracking of the object requires huge features
which are not possible most of the times [9].
To enhance the performance, CNN is utilized for classification and recognition of the image. CNN
includes lakhs of different images of different class [10-11]. CNN is used to extract information of image and
to study complicated features. Algorithm for object tracking based on CNN with the architecture of shift
variant is used to get features we studied. The transitory features are being studied during online process. The
behaviors such as temporal and spatial are considered by a couple of images in place of an image [12].
R. Phadnis et al. [13] did involve approach, where the results of last year of pre-set CNN module are
cascaded with online SVM to study different appearance systems. Bayesian network targeting is used for
particular saliency map thus making the algorithm very much complicated for just detecting the vehicles. The
algorithms work for the given set of datsets under a condition but when considered for different climatic
condition the accuracy will have major affect. Hence, we have proposed an algorithm which works in all
given climatic conditions [14-21].
The brief introduction and contributions of the model is as listed: (a) A method is put forth to detect
vehicles based on tensorflow and corner points of the vehicle and double verifying the detected object with
upholding the accuracy of the proposal. (b) The proposed model use corner points which help to detect the
object anywhere irrespective of the climatic changes.

2. RESEARCH METHOD
Architecture of the proposed system is shown in Figure 1. From the architecture one can make out
that vehicle detection is performed using tensorflow. Once the vehicles are detected the corner points are
detected within the vehicle region. The detected corner points help in verifying whether the detected object is
vehicle or no. After the detection of the vehicles, the interested points from the vehicular region are passed to
CNN algorithm which helps in tracking.

Frame Vehicle Detection Corner Points Vehicle Tracking

Using Tensorflow using CNN

Figure 1. Flow Architecture of the proposed algorithm

In this system video is considered as an input. In the next step frames are considered for processing.
Deep learning methods are used in detection of object and it’s tracking algorithms. The detection of object is
explained in depth in different conditions like light variation, illumination using tensorflow [22].
The object detection API based on tensorflow is a platform of open source, it is easy to build, train
and detect and it is built on top of tensorflow. This detection of object which is based on tensorflow is
proposed in Figure 2. In proposed method firstly the libraries which are necessary are imported. Later the

Vehicle detection and tracking for traffic management (Mallikarjun Anandhalli)

68 ISSN: 2252-8938

system which is pre-trained for object detection is imported, along with box and tensor class weights are
initialized.
After the initialization of all the parameters of the tensorflow model, the detected object in the
image is taken. The tensorflow is used to detect vehicles, once the detection of vehicles is done using
tensorflow then the corners present the image are calculated which will help us in double verification of the
detected vehicles. The Figure 2 shows the flow chart of the model.

Figure 2. Flow diagram of the proposed algorithm

The tensorflow model which is loaded from the trained model is applied on the image, the image is
tested by this model and then the object’s location in the image is returned. This method is based on the
algorithm of tensorflow object detection. The accuracy of this method is better and can be applied to RGB
images. To initiate the tracking process objects locations are needed. In this method CNN based tracking
algorithm is used instead of convolutional computer vision-based algorithm.

2.1. Corner points

From the detailed analysis of vehicular region in an image after the corners are detected it is
found that the corners are densely packed with in the vehicular region and can be seen in the following
Figure 3(a-b). Once the detected vehicles are verified then the centroid of the object or the interest points of
the objects are passed to the CNN. The CNN will help in reliable tracking of the vehicles even though the
occlusion, shadows appear in the image [23-24].
The λ1 and λ2 are the Eigenvalues which are calculated from the matrix M computed from image
derivatives Ix and Iy for the detection of the corner points. Based on these Eigenvalues pixels are classified
and they are:
1. If λ2 >> λ1 or λ1 >> λ2 Then the pixel is edge
2. If λ1 and λ2 are larger than it is corner
3. If λ1 and λ2 are small than it is flat
The corner response is calculated by

𝑅 = det 𝑀 − 𝑘(𝑡𝑟𝑎𝑐𝑒 𝑀)2

det 𝑀 = λ1 λ2
trace 𝑀 = λ1& λ2

Int J Artif Intell, Vol. 10, No. 1, March 2021: 66 – 73

Int J Artif Intell ISSN: 2252-8938 69

Where k is constant and varies from 0.04 to 0.06

Response is large for a corner and it only depends on M

(a) (b)

Figure 3. Results of corner detection, (a) Original input file, (b) Output of corner detection

2.2. Grouping of corner points

The corner points that lie in the non-vehicular region are removed as they lie in non region of
interest. The points are densely packed within the region of interest hence these points are grouped using
Euclidian distance and is given as

𝑓 = √(𝑥2 − 𝑥1 )2 + (𝑦2 − 𝑦1 )2

Before grouping of points a threshold value is set. This value is nothing but the maximum distance
between any two points in an image to group them as one and also each group should have minimum number
of corner points to consider the object has vehicle. The threshold value varies with respect to the resolution of
videos, and camera. If the distance between the points is less than the threshold, they are discarded. In this
way corner points are grouped accordingly which in turn helps in re-verification of the object.
If the distance between any two points is less than the threshold value set the two points are grouped
as one. In this way, set of random interest points are grouped based on the threshold value. Each group of
points correlates with that of a vehicle.

2.3. CNN
The algorithm related to the tracking of the object plays a very important role in computer vision
area. Understanding of homogenous motion and analyzing variation caused over time and information of the
object are needed for accurate tracking of an object. Tracker should be able to adopt for any new
observations and model them accordingly. The model is trained with the pre-defined weights. The model has
the ability to include the temporal information. The pre-set model which is trained on huge number of objects
in real time is used instead of concentrating on the objects during testing time. The object at the speed of
150fps can be tracked by these lightweight systems. Also, it has ability to remove the occlusion problem.
And in the proposed work, the locations of the objects which are achieved from the algorithm of object
detection based on tensorflow are passed to the algorithm of object tracking based on CNN. First the
positions are studied by the model and are searched within the frames.
Although CNNs use max-pooling layers to allow CNNs to be spatially-invariant for the data that is
being given as the input, the feature maps which are intermediate are not actually invariant to the
transformations which are large of the input data. Hence, we not only classify the patch into object but also
classification of the location of the object in the patch or basically tracking is accomplished. Instead of
generating random patches that cover the region of interest (ROI), here we generate fixed-spacing random
patches that cover the ROI. It also reduces the risk of missing the target. Here, instead of passing each patch
to the convolutional layer for classification, the whole ROI is passed through the layer, which would save the
redundant computations. Taking the mean location of the patches the target location is determined. Bilinear
interpolation can be used for object detection applications.

Vehicle detection and tracking for traffic management (Mallikarjun Anandhalli)

70 ISSN: 2252-8938

3. RESULTS AND DISCUSSION

In normal situations, all the state of art algorithm has shown good results but in other environmental
conditions like rain, fog, snow it is difficult to get accurate and good results. It may fail to either to detect
vehicle or track the vehicle and hence motion calculation of the object leads to varying results. The proposed
algorithm is effective even in complex conditions such as trivial condition such as shadows. The reason for
this is due to the corner features which are shift and rotational invaraiant. Even under less illumination the
corner points are detected. The qualitative results were obtained through the evaluation process of ground
truth image and algorithm output files as input hence result can be accurate and average. The testing of
algorithm is done on different videos in different environmental conditions. The datsets were collected from
web.
The results of algorithm shown in Figure 4(a-b) are sunny with low resolution videos even though
the tensflow miss the detection of the algorithm, but the corner points help in detection of the objects as
vehicles. The detection rate of the tensorflow is 90% on the standard datasets. But whereas the detection rate
of the proposed algorithm is almost equal to 90%, the tabular column defines these parameters.

(a) (b)

Figure 4. Results of vehicle detection under low resolution sunny day, (a) First example, (b) Second example

The results shown in the Figure 5(a-b) are cloudy and sunny with good resolution videos and can be
seen that the algorithm detects all of the vehicles irrespective of the distance. On glance the Figure 5(b) and
Figure 4(b) one can make out that the cyclist is not detected as a vehicle. The main reason for this is that
vehicles do pose a good number of corner points and where as non vehicle region do not. This analysis of the
corner points have really helped to achieve higher accuracy on all types of datasets.

(a) (b)

Figure 5. Results of vehicle detection under different climatic condition, (a) Cloudy Condition, (b) Sunny day

The results shown in the Figure 6(a-b) are rainy low and good resolution videos and can be seen that
the algorithm detects all of the vehicles irrespective of the distance. The basic property of the ir-rotational,
shift-invariant due to these properties the vehicles can be detected even in shadow regions and rainy regions.

Int J Artif Intell, Vol. 10, No. 1, March 2021: 66 – 73

Int J Artif Intell ISSN: 2252-8938 71

The reason is the corner points pose under any of these conditions and thus helping in detection of the
vehicles.

(a) (b)

Figure 6. Results of vehicle detection under different climatic condition, (a) Rainy condition, (b) Shadow
condition

Based on result analysis made considering different environmental scenario the complete anaylsis
has been put in the Table 1. In the Table 1, the resulotion of the corresponding dataset has been considered
for the study along with FPS, Climate, No. of vehicles present vs vehicles detected along with missed
detection and false detections as been listed in the Table 1 along with the corresponding results.

Table 1. Results of proposed model

Vehicles No. of vehicles Missed Falsely detected
Sl.No. Resolution Fps Climate
Detected Present (TP) vehicles (FN) vehicles (FP)
1 320*240 10 Sunny(low resolution) 4 4 0 0
2 320*240 15 Sunny(low resolution) 5 5 0 0
3 640*480 25 Cloudy 8 8 0 0
4 320*240 30 Sunny (High resolution video) 4 4 0 0
5 704*480 30 Rainy 8 8 0 0
6 704*480 30 Sunny Complex (Multiple) 5 4 0 1

3.1. Performance
The performance of the model is compared with state of art systems. The parameters to measure the
performance of the model are correctness; completeness and quality which are explained by Wiedemann and
are listed is [10]:
True positive (TP): Vehicles accurately detected as vehicles.
False positive (FP): Vehicles incorrectly found as vehicles.
False pegative (FN): Vehicles wrongly identified as non-vehicles.
𝑇𝑃 𝑇𝑃 𝑇𝑃
𝐶𝑜𝑟𝑟𝑒𝑐𝑡𝑛𝑒𝑠𝑠 = 𝑇𝑃+𝐹𝑃*100% 𝐶𝑜𝑚𝑝𝑙𝑒𝑡𝑒𝑛𝑒𝑠𝑠 = 𝑇𝑃+𝐹𝑁*100% 𝑄𝑢𝑎𝑙𝑖𝑡𝑦 = 𝑇𝑃+𝐹𝑁+𝐹𝑃*100%

3.2. Computation effectiveness

Opencv [19] tool along with C++ is used to implement the algorithm. The computational cost of the
system is 105ms to process a frame which is considerable in real world scenario.

3.3. Comparative analysis

The states of art algorithm [1, 25-26] are considered for the comparison with the model based on the
results obtained. The result of the algorithm is more reliable and is simple, effective and time efficient.
Accuracy of the system is 92% in multi view, in side view and top view the algorithm is even better with
92% because we can easily differentiate the colors with the surrounding environment, in front and rear view
the accuracy is 92% which is better than the other results and can hold for all the views. On comparing the

Vehicle detection and tracking for traffic management (Mallikarjun Anandhalli)

72 ISSN: 2252-8938

accuracy of proposed algorithm with other three state of art works with respect to there detection view the
proposed algorithm has 92% accuracy in detection of the vehicle as shown in Table 2.

Table 2. Comparison of the results

View Accuracy

Algorithm Side View and Top View Front and Rear View Multi View
Proposed Algorithm 97% 91% 92%
Combining Multiple Parts for Complex Urban Surveillance [1] X 92% X
Morphological operations [26] 91.5% X X
Low-Exposure Color Video for Night Conditions [25] 89% X X

4. CONCLUSION
In proposed paper, tensorflow fused with corner points are taken into the consideration for the
detection of the objects as vehicles. From the region of Interest, the points are passed to the convolutional
neural network which in turn is used for vehicle tracking. The proposed algorithm for the vehicle detection
detects all the vehicles present in the any of the scenario under different climatic conditions and the results in
terms of its accuracy and time taken are very good. The tracking algorithm based on CNN is used for
tracking the detected vehicle. The parameters which were set for the proposed algorithm, which is the
sensitivity with detection rate of 92%, similarly accuracy of about 91.8%, specificity of about 91.2%.

ACKNOWLEDGEMENTS
This research project is funded by Science and Engineering Research Board (SERB), Government of
India and is supported by KLS Gogte Institute of technology, Belagavi.

REFERENCES
[1] Bin Tian, Ye Li, Bo Li, Ding Wen, “Rear-View Vehicle Detection and Tracking by Combining Multiple Parts for
Complex Urban Surveillance,” in Intelligent Transportation Systems, IEEE Transactions on, vol.15, no.2, pp.597-
606, April 2014. DOI: 10.1109/TITS.2013.2283302.
[2] Ye Li, Bo Li, Bin Tian, Qingming Yao, "Vehicle Detection Based on the and-or Graph for Congested Traffic
Conditions," in Intelligent Transportation Systems, IEEE Transactions on, vol.14, no.2, pp.984-993, June 2013.
DOI: 10.1109/TITS.2013.2250501.
[3] Anandhalli, M., Baligar, V.P., "Improvised approach using background subtraction for vehicle detection,"
in Advance Computing Conference (IACC), 2015 IEEE International, vol., no., pp.303-308, 12-13 June 2015. DOI:
10.1109/IADCC.2015.7154719.
[4] Ye Li, Bo Li, Bin Tian, Fenghua Zhu, Gang Xiong, Kunfeng Wang, "Vehicle detection based on the deformable
hybrid image template," in Vehicular Electronics and Safety (ICVES), 2013 IEEE International Conference on,
pp.114-118, 28-30 July 2013. DOI: 10.1109/ICVES.2013.6619614.
[5] F. Yamazaki, W. Liu, “Vehicle extraction and speed detection from digital aerial images,” in Proc. IEEE IGARSS,
2008, vol. 3, pp. 1334-1337. DOI: 10.1109/IGARSS.2008.4779606.
[6] S. Li, H. Yu, J. Zhang, K. Yang and R. Bin, "Video-based traffic data collection system for multiple vehicle types,"
in IET Intelligent Transport Systems, vol. 8, no. 2, pp. 164-174, March 2014. doi: 10.1049/iet-its.2012.0099.
[7] O'Malley, R. Jones, E., Glavin, M., "Rear-Lamp Vehicle Detection and Tracking in Low-Exposure Color Video for
Night Conditions," in Intelligent Transportation Systems, IEEE Transactions on, vol.11, no.2, pp.453-462, June
2010, doi: 10.1109/TITS.2010.2045375.
[8] H. Samet, M. Tamminen. "Efficient Component Labeling of Images of Arbitrary Dimension Represented by Linear
Bintrees," IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume: 10, Issue: 4, Jul 1988. DOI:
10.1109/34.3918.
[9] R.E. Burkard, M. Dell'Amico, S. Martello: Assignment Problems (Revised reprint). SIAM, Philadelphia PA. 2012.
[10] C. Wiedemann, C. Heipke, H. Mayer, and O. Jamet, “EmPirical evaluation of automatically extracted road axes,”
in EmPirical Evaluation Methods in Computer Vision, K. Bowyer and P. Phillips, Eds. New York, NY, USA: IEEE
Comput. Soc. Press, 1998, pp. 172-187.
[11] M Anandhalli, VP Baligar, “An Approach to Detect Vehicles in Multiple Climatic Conditions Using the Corner
Point Approach”, Journal of Intelligent Systems, 27(3), pp. 363-376. 2018. DOI: 10.1515/jisys-2016-0073.
[12] Won Jin Kim, In-So Kweon, “Moving object detection and tracking from moving camera,” 2011 8th International
Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Incheon, pp. 758-759, 2011. DOI:
10.1109/URAI.2011.6146005.
[13] R. Phadnis, J. Mishra and S. Bendale, "Objects Talk-Object Detection and Pattern Tracking Using
TensorFlow," 2018 Second International Conference on Inventive Communication and Computational
Technologies (ICICCT), Coimbatore, 2018, pp. 1216-1219, doi: 10.1109/ICICCT.2018.8473331.

Int J Artif Intell, Vol. 10, No. 1, March 2021: 66 – 73

Int J Artif Intell ISSN: 2252-8938 73

[14] Zhang Zhisheng, Han Yanxiang, Zhou Yifan, et al., “A novel absolute localization estimation of a target with
monocular vision,” Optik-Int J Light and Electron Optics., 124(12):1218-1223, 2013. DOI:
10.1016/j.ijleo.2012.03.032.
[15] Aswini N, Uma SV. “Obstacle avoidance and distance measurement for unmanned aerial vehicles using monocular
vision,” Int J Electr Comput Eng. (IJECE), 9(5), pp. 3504, 2019. http://doi.org/10.11591/ijece.v9i5.pp3504-3511.
[16] Huang Liqin, Chen Yanan, Fan Zhengjia, et al., “Measuring the absolute distance of a front vehicle from an in-car
camera based on monocular vision and instance segmentation,” J Electron Imaging, 27(04):1, 2018. DOI:
10.1117/1.JEI.27.4.043019.
[17] Wei S-D, Lai S-H. “Fast template matching based on normalized cross correlation with adaptive multilevel winner
update,” IEEE Trans Image Process, 17(11), pp. 2227-2235, 2008. DOI: 10.1109/TIP.2008.2004615.
[18] Slimani I, Zaarane A, Hamdoun A, Atouf I. “Vehicle License Plate Localization and Recognition System for
Intelligent Transportation Applications,” 6th International Conference on Control, Decision and Information
Technologies (CoDIT), 2019. DOI: 10.1109/CoDIT.2019.8820446.
[19] Al OW, Zaarane A, Slimani I, Atouf I, Benrabh M., “Vehicular queue length measurement based on edge detection
and vehicle feature extraction,” J Theor Appl Inf Technol, 97(5):1595-1603, 2019.
[20] C. Wolf, J.-M. Jolion, “Object count/area graphs for the evaluation of object detection and segmentation
algorithms,” Int. J. Doc. Anal. Recognit., vol. 8, no. 4, pp. 280-296, 2006. DOI: 10.1007/s10032-005-0014-O.
[21] M. Everingham, L. V. Gool, C. K. I.Williams, J.Winn, and A. Zisserman, “The PASCAL Visual Object Classes
(VOC) challenge,” Int. J. Comput. Vis., vol. 88, pp. 303-338, 2009. https://doi.org/10.1007/s11263-009-0275-4.
[22] D. G. R. Bradski, A. Kaehler, Learning OpenCV, 1st ed. Sebastopol, CA, USA: O’Reilly Media, 2008.
[23] Anandhalli, M., Baligar, V. P., “An Approach to Detect Vehicles in Multiple Climatic Conditions Using the Corner
Point Approach,” Journal of Intelligent Systems, 27(3), 363-376, 2018.
[24] Anandhalli, Mallikarjun, Baligar, Vishwanath P, “A novel approach in real-time vehicle detection and tracking
using Raspberry Pi”, Alexandria engineering journal, 57(3), 1597-1607, 2018 Elsevier.
[25] Renjun Lin, Xianbin Cao, Yanwu Xu, Changxia Wu, Hong Qiao, "Air-borne moving vehicle detection for video
surveillance of urban traffic," in Intelligent Vehicles Symposium, 2009 IEEE, pp.203-208, 2009. DOI:
10.1109/IVS.2009.5164278.
[26] Zezhong Zheng, Guoqing Zhou, Yong Wang, Yalan Liu, Xiaowen Li, Xiaoting Wang, Ling Jiang, "A Novel
Vehicle Detection Method with High Resolution Highway Aerial Image," in IEEE Journal of Selected Topics in
Applied Earth Observations and Remote Sensing, vol.6, no.6, pp.2338-2343, 2013. DOI:
10.1109/JSTARS.2013.2266131.

Vehicle detection and tracking for traffic management (Mallikarjun Anandhalli)