Computer Science & Engineering: An International Journal (CSEIJ), Vol 12, No 3, June 2022

VEHICLE DETECTION AND COUNT IN THE

CAPTURED STREAM VIDEO USING OPENCV IN
MACHINE LEARNING
Md.Milon Rana1, Tajkuruna Akter Tithy1 and Md. Mehedi Hasan2
1
Department of Electronics and Communication Engineering, Hajee Mohammad Danesh
Science and Technology University, Dinajpur, Bangladesh
2
Department of Pharmacy, World University of Bangladesh, Dhaka, Bangladesh

ABSTRACT
The technology of detection within the captured video has implementation within the sort of fields. This
emerging technology when implemented over the real-time video feeds could even be beneficial. The
supreme good thing about vehicle detection within the real-time streaming video feed is to trace vehicles in
busy roads or Bridges like Padma or Jamuna Bridge. An accidents occurred anywhere which may rather
be detected. Vehicle detection also called computer vision beholding, basically the scientific methods and
ways of how machines see instead of human eyes. This chapter aims to explore the prevailing challenging
issue within the planet of unsupervised surveillance and security, Helps traffic police, Maintaining records
and Traffic surveillance control. The detection of vehicles is implemented with enhanced algorithms and
machine learning libraries like OpenCV, TensorFlow, and others. The varied approaches are accustomed
identify and track the particular object through the trained model from the captured video.

KEYWORDS
Image classifications, Video tracking, Information analysis, Vehicle detection, Vehicle count

1. INTRODUCTION
Since the population and transport system increase day by day, the demand for managing them
increase at the identical time. the globe is getting populated so fast. Therefore the quantity of
machines from any types including vehicles increased at the identical time. That being said, new
topics like traffic, accidents and plenty of more issues are needed to be managed. it's hard to
manage them with the old methods, new trends and technologies are found and invented to
handle each and each milestone that human kind is trying achieve. one amongst these challenges
is traffic in highways and cities. Many options like light, sign, etc. deployed so as to cope with
this phenomena. It seems that these options aren't enough or not so efficient alone. New
technologies like object detection and tracking are invented so as to utilize automated camera
surveillance to provide data that may give meanings for a choice making process. This
phenomena are used for various quite issues. The new trend Intelligent Transport System (ITS)
has many elements which object detection and tracking is one among them. this technique is
employed to detect vehicles, lanes, traffic sign, or vehicle make detection. The vehicle detection
and classify ability gives us the likelihood to enhance the traffic flows and roads, prevent
accidents, and registering traffic crimes and violations.

Objective of this chapter briefly presents the methodology for the detection and count of a
specific variety of vehicle within the streaming video. This work is required a particular

DOI:10.5121/cseij.2022.12301 1
 Computer Science & Engineering: An International Journal (CSEIJ), Vol 12, No 3, June 2022
mechanism within the surveillance and security. Efficient identification of a vehicle in live
stream video requires massively enhancement within the surveillance techniques to counter the
terrorism. In high speed stream handle, the live video stream is additionally simultaneous copied
to the storage server to explore the insights into the chunk of video frames for further process [1–
3]. This object detection and identification within the captured video aims to acknowledge the
moving vehicle within the chunk video frames to track the movement of that exact target in
whole video stream [4].

Machine learning technologies are successfully employed in the face detection and face
recognition. The video object co-segmentation is a few task of computer vision in which it may
be widely used. the precise styles of the vehicle count help to reinforce the surveillance technique
for the captured live stream. It are often finished the assistance of various machine learning
technologies using Python like OpenCV and Tensor- Flow. Computer vision may be employed in
object detection, object tracking, object classification, video surveillance, and background
modelling [2]. we are able to consider numerous examples, tracking of a football and therefore
the cricket equipment within the football and cricket matches, respectively. On the opposite hand,
it's to acknowledge the people within the stream video or to detect the movement of car, bike, and
truck within the stream. That is an example to detect a particular target in surveillance video
footage.

The vehicle detection within the streaming video also aims to resolve some real-time problem,
for example, the difficulty that video feeds can not be processed in real [5, 6]. We concentrate on
the vehicle detection within the captured video stream in order that the track of the overall
vehicles can be maintained and also the total moving vehicles within the captured video stream
can be calculated [7]. Further, we will explore the phases like classification of the vehicles
supported light and heavy vehicles or two-wheelers and four-wheelers or based on brands. The
classification of the moving vehicles can do using the artificial neural network, AdaBoost
algorithm, and support vector machine.

2. OVERVIEW OF THE APPROACH FOR VEHICLE DETECTION

The vehicle detection within the captured video stream is finished with the assistance of the
OpenCV through which the count of the entire number of moving vehicle is being maintained.
The initial approach is to detect and count the entire number of moving vehicles within the
captured video stream with the assistance of ML libraries like OpenCV. The other approach is to
coach the model to acknowledge a selected object with the assistance of ML libraries like
TensorFlow. Further, both approaches are to be combined to spot and track the precise object
through the trained model. The approach is employed to make a model with the assistance of two
different ML approaches. The initial approach is that the implementation of the OpenCV library.
With the assistance of this library, it becomes easier to maintain the record of the moving
vehicles within the video stream which is further targeted to be achieved with the assistance of
the training of the model. the idea of the second approach is that the implementation about the
Inception-ResNetV2. it's one in every of the fastest algorithms available now.[12]

When AlexNet was presented, it absolutely was discovered that deeper neural networks are
needed for further classification. However, processing it takes longer. So, it's trade-off between
accuracy and speed. Google. Inc came up with new approach and divides the model in numbers
of modules. This approach is also employed in the Inception-ResNetV2. It takes the input image
of size 299*299*3 in stem cell of the model. it's further divided into two branches then that the
processing may be divided. After that, the results are concatenated. To detect the article,
primarily it is considered the Python-based machine learning libraries of TensorFlow, OpenCV,

2
 Computer Science & Engineering: An International Journal (CSEIJ), Vol 12, No 3, June 2022
Keras, Pandas, sklearn, Matplotlib, JSON, and PIL. InceptionV4, Inception-ResNet, NumPy,
Itertools, and shutil are getting used to attain the target output from this phase.

Vehicle detection is very important for several fields, like military, civilian, and government
applications. the govt needs real-time automatic vehicles relying on every road to manage the
traffic. It either can use streams or pre-recorded video as classification and counting image
objects.

2.1. Vehicle Detection and Counting in Traffic Video Based on OpenCV

With the trendy social economy increasing, vehicle number in countries is growing rapidly,
including China. Therefore, getting real-time traffic parameters vital thanks to the limited road
space significance. Nowadays, the vehicle video detection method supported image processing
develops rapidly. The project used real-time traffic video and processed to induce each frame.
Then the video was filtered by the Gaussian denoising. After that, it marked the region of interest
(ROI) and applied a background subtraction algorithm. Next, they get the binarization foreground
image, set threshold to eliminate moving objects (whose space is simply too small), and check
the boundary of ROI to judge the moving vehicle and count. These steps are wont to get
parameters results of intelligent transportation.

2.2. Video Tracking

The secondary functionality is employed to trace and record the video frames. It labels
executions, throughout the thing era. within the current scenario, the model involves the detection
of vehicles and records the whole count of the moving vehicles in an exceedingly stable video
stream. Real-time computer vision are often achieved with the assistance of OpenCV library,
which is an open-source computer vision library. The OpenCV library includes support vector
machine-based algorithms, artificial neural network based algorithms, K-nearest neighbour
algorithm, and plenty of others.

3. METHODOLOGY
This research will use computer vision and machine learning [8] to process image data. we'll
implement Computer vision and machine learning for various user needs, including classifying
and counting the quantity of vehicles recorded by the camera; the processed object may be a
video or a photograph. In computer vision, we will use several steps within the OpenCV [9][10]
library within the coding process. Opening models and classes (1), reading input (2), and
processing frames (3). After that, we are able to use mask R-CNN [11] to predict whether the
image object has met the factors or not. Basically YOLO V-3 is used to this detection & counting
process. YOLO stands for You Only Look Once. It is a real-time object recognition algorithm.
It can classify and localize multiple objects in a single frame. YOLO is a very fast and accurate
algorithm for its simpler network architecture.

3.1. How does YOLO Work

YOLO workshop using substantially these ways.

1. Residual Blocks – principally, it divides an image into NxN grids.

2. Bounding Box retrogression – Each grid cell is transferred to the model. also YOLO
determines the probability of the cell contains a certain class and the class with the maximum
probability is chosen.

3
 Computer Science & Engineering: An International Journal (CSEIJ), Vol 12, No 3, June 2022
3. crossroad Over Union( IOU) – IOU is a metric that evaluates crossroad between the
prognosticated bounding box and the ground verity bounding box. ANon-max repression fashion
is applied to exclude the bounding boxes that are veritably close by performing the IoU with the
one having the loftiest class probability among them.

Figure 1. YOLO Architecture

In this chapter, all methods and tools wont to detect and track vehicles are being discussed. At the
very beginning, tools that are utilized within the research is described in details that how they're
getting used within the context. After that, the implementation process started with data cleaning,
preprocessing and algorithms are presented with a brief summary of those issues at the top of the
chapter.

3.2. Tools Used

Like every other researches and studies this thesis utilized some applications and tools for
creating models and experiments. For such researches many tools needed to be used, for instance
the author used python because the artificial language of the model development or the dataset
with plenty of auto and non-vehicle images used for training or as an example many python
libraries which are necessary or we will say useful so as to make ML models. during this chapter
all the tools that are being employed within the thesis are presented.

3.3. Python

Python may be a general purpose programing language which is made by Guido van Rossum, it's
used for various platforms like mathematic, computer GUI, web so many large scientific
applications. the most aim creating python was to form programming easy in order that everyone
within the world is ready to put in writing code. Therefore it’s popular for its simplicity. Python
is sensitive in spacing. Despite the very fact that python was created for youths, for the present it
bits all programming languages in many fields which is amazing and somehow unbelievable.
Python is ready to try to to whatever other programming languages can do. From web to
algorithms to desktop applications etc. within the past few years’ python became popular in in AI
and machine learning applications thanks to having many efficient and handy libraries which
makes the task much easier and faster. Experts coming to the present field have various
backgrounds. Since they don’t have programming background, the foremost easy and convenient
language to start out with is python. 21 Despite the above mentioned properties, the researcher’s

4
 Computer Science & Engineering: An International Journal (CSEIJ), Vol 12, No 3, June 2022
knowledge and self-interest caused to decide on python because the artificial language of the
model development. The libraries that are utilized during this thesis are as follows.

3.4. Numpy

Numpy is an open source library, it does the computing with the assistance of multi-dimensional
matrices and arrays. It contains variety of functions which makes it easy to figure with these sort
of data. In data analysis if we would like to create the speed fast and efficient we want to use
arrays, therefore this library helps data scientists to figure faster with large amounts of
information. Usually for detection and forecasting, the models function needs arrays as parameter
to work fast and reduce the training prediction time.

3.5. Matplotlib

Plotting is growing altogether fields to visualise the information and to form it understandable.
Therefore Matplotlib is being employed as a plotting library to form different quite graphs and
figures for sort of aims. the nice thing about matplotlib is that it can produce good plots and
graphs with just few lines of codes. So matplotlib is employed to extract color features and build
histograms.

3.6. Jupyter Notebook

Jupyter Notebook may be a web based application which makes us ready to create and modify
live codes, equations, plaintexts and visualizations. this can be an open source notebook which
supports many programming languages. this can be used for various purposes like machine
learning, numerical simulation, information visualization etc. the researcher decided to use this
notebook for writing readable codes and implementing machine learning algorithms.

3.7. Computer

The PC that's being employed to coach and test the models has the subsequent properties: Model:
Dell inspiron 13-5378 RAM: 8 GB Processor: Core i7 Quad Core Graphic: Intel HD 4 GB

3.8. OpenCV

OpenCV (Open Source Computer Vision Library) is an open source computer vision and
machine learning software library. OpenCV was built to provide a common infrastructure for
computer vision applications and to accelerate the use of machine perception in the commercial
products.

4. IMPLEMENTATION AND RESULTS

The objective of this research is to count every vehicle which is ready to enter the road. Every
vehicle that will enter the lot will pass a camera installed at the doorway. The tactic utilized in
counting it passing through the parking gate is that the identical because the strategy accustomed
classify and count it on the road. This study's video sample is additionally a video of vehicles'
flow on the road with various vehicle types at different speeds. In this paper, the results exposed
that in a road many vehicles are travelled from one part to another. For counting how many
vehicles are travelled in a day or in a week we can calculate. Basically it is important to detect
vehicles and counting. In a road a traffic police can’t count how many vehicles are passing in a
day. For the helping purpose if we use this system in any road then we count how many vehicles

5
 Computer Science & Engineering: An International Journal (CSEIJ), Vol 12, No 3, June 2022
are passing in this road and which time as a result we can determine and detect accident or any
types of crime if these vehicles occurred.

4.1. Object Detection

We use recorded traffic on the underpass road in Yogyakarta province as a pursuit sample. Then
we use these videos for vehicle classification and calculation implementation, as shown in Figure

After being loaded, we create a detection box to spot the counting area (red box). After that, we
create a bounding box around each object (green box). the item detector computes every enclose
the video. Any object detector can produce bounding boxes, like color thresholding + contour
extraction, SSDs, R-CNNs, etc. This paper uses mask R-CNN to compute object detection and
put the bounding box around an object.

4.2. Tracking & Counting

Once we've got the bounding box, we've to be compelled to calculate the "centroid." The centroid
is that the center coordinates (x, y) of the bounding box. It uses to trace the item movement
within the video. Whether the item is passing through the imaginary red box or not. As the
vehicles fade, the system will automatically classify the vehicle, count them one by one, then
show us the results, as shown in Figure

6
 Computer Science & Engineering: An International Journal (CSEIJ), Vol 12, No 3, June 2022

Figure 2. Classification and counting

5. CONCLUSION
In this chapter vehicle detection, methodology is presented for object detection for the video
surveillance. We have presented our model on InceptionV4 and Inception- ResNet that classify
images and label videos supported the training it received. It has many applications within the
field of medical diagnoses, defence, games, and virtualization and has high accuracy and high
speed compare to other models presented up to now. Video detection within the captured video
aims to acknowledge the moving vehicle within the captured video stream, to trace the movement
of that concentrate on in whole video, and to identify the full count of the moving vehicles within
the captured video. this is often exhibited with the assistance of OpenCV and TensorFlow. This
includes background subtraction; it is a way of eliminating the background from image. to attain
this, we extract the moving foreground from the static background. Afterwards, the thresholding
is completed to achieve contrast, then noise removal is completed with the assistance of
morphological transformation.

Object detection is completed through contours then finally tracking, and total counting of the
moving vehicle is finished. Identification of the item is completed through TensorFlow. This
work was tested on the ball but is trained on unusual data sets of about 1500 images and may be
used widely utilized in different fields. There are many problems occurred during the running of
our module for vehicle detection. The initial problem is while capturing the video. There mustn't
be any noise. If the video isn't still, then the OpenCV cannot identify the moving object precisely
and due to whole frame moving, it'll detect multiple still object because the moving object.

In the night light, the moving vehicles can not be determined easily within the streaming video.
For this purpose, we want to own the night-vision camera for the precise capturing of the moving
vehicles within the streaming night light video. within the training of the multiple images within
the Google software named Colab took several hours. This process is that the time taking
process, which gave the most effective results when trained over the multiple large input images,
and that we will get good accuracy, e.g. for 1500 images files the accuracy is with reference to
80%. For the smaller inputs, the accuracy is additionally tinier. And the same is to be in deep
trouble the videos also. The optimization of the model is to be done in order that the assorted
errors are often corrected to attain the required output. We can apply various optimization
techniques like feature extraction during detection phase.

Regards during this case, we will use the canny edge detector with moment preserving
thresholding. Noise minimization is finished the assistance of dynamic Bayesian network (DBN).
7
 Computer Science & Engineering: An International Journal (CSEIJ), Vol 12, No 3, June 2022
during this proposed method, it's better to use Bayesian network (BN), which is taken into
account for pixel-wise classification. It classifies the pixels of vehicles and non-vehicles.

6. FUTURE WORKS
In future we can try our best to improve this system. Now it can detect any type of vehicles. In
future we try to detect specific vehicles like cars, Buses, Trucks, Motor Bike and so on. And also
try to detect the speed of these vehicles.

REFERENCES

[1] S. Kanrar, M. Siraj, “Performance of multirate multicast in distributed network.” Int. J. Commun.
Netw. Syst. Sci. 3(6), 554–562, (2010) https://doi.org/10.4236/ijcns.2010.36074
[2] S. Kanrar, N.K. Mandal, Video traffic analytics for large scale surveillance. Multimedia Tools Appl.
76(11), 13315–13342 (2017). https://doi.org/10.1007/s11042-016-3752-0
[3] S. Kanrar, N.K. Mandal, Video traffic flow analysis in distributed system during interactive session,
Adv. Multimedia. 2016, Article ID 7829570, pp 1–14 (2016), https://doi.org/10.1155/2016/7829570
[4] J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell.
PAMI 8(6), 679–698 (1986)
[5] Minkyu Cheon, Wonju Lee, Changyong Yoon, Vision-based vehicle detection system with
consideration of the detecting location. IEEE Trans. Intell. Transp. Syst. 13(3), 1243–1252 (2012)
[6] B.F. Lin, Y.M. Lin, Fu Li-Chen, P.-Y. Hsiao, L.-A. Chuang, S.-S. Huang, M.-F. Lo, integrating
appearance and edge features for sedan vehicle detection in the blind-spot area. IEEE Trans. Intell.
Transp. Syst. 13(2), 737–747 (2012)
[7] C.C.R. Wang, J.J.J. Lien, “Automatic vehicle detection using local features—a statistical approach,”
9(1), 83–96 (2008)
[8] G. Meena, D. Sharma, and M. Mahrishi, “Traffic Prediction for Intelligent Transportation System
using Machine Learning,” Proc. 3rd Int. Conf. Emerg. Technol. Comput. Eng. Mach. Learn. Internet
Things, ICETCE 2020, no. February, pp. 145–148, 2020.
[9] D. Li, B. Liang, and W. Zhang, “Real-time moving vehicle detection, tracking, and counting system
implemented with OpenCV,” ICIST 2014 - Proc. 2014 4th IEEE Int. Conf. Inf. Sci. Technol., pp.
631–634, 2014.
[10] A. P. Kulkarni and V. P. Baligar, “Real Time Vehicle Detection, Tracking and Counting Using
Raspberry-Pi,” 2nd Int. Conf. Innov. Mech. Ind. Appl. ICIMIA 2020 - Conf. Proc., no. Icimia, pp.
603–607, 2020.
[11] W. Cai, J. Li, Z. Xie, T. Zhao, and K. Lu, “Street object detection based on faster R-CNN,” Chinese
Control Conf. CCC, vol. 2018-July, pp. 9500–9503, 2018.
[12] F. Ekonomi, J. Akuntansi, and U. S. Ratulangi, “Analisis Efektivitas Pajak Parkir Dan Kontribusinya
Terhadap Pendapatan Asli Daerah ( Pad ) Kota Kotamobagu Analysis of the Effectiveness of Parking
Tax and its Contribution to Regional Original Income ( Pad ) Kotamobagu City Jurnal Emba,” J.
Emba J. Ris. Ekon. Manajemen, Bisnis Dan Akunt., vol. 8, no. 4, pp. 381–388, 2020.

8
 Computer Science & Engineering: An International Journal (CSEIJ), Vol 12, No 3, June 2022
AUTHORS

Md. Milon Rana received the B. Sc. (Engineering) from the department of Electronics
and Communication Engineering (ECE), Hajee Mohammad Danesh Science and
Technology University (HSTU), Dinajpur-5200, Bangladesh in 2019. Currently, He is a
student of M. Sc. (Engineering) in the same department. His research interest is
performance analysis of communication protocols of IoT, Machine Learning, Deep
Learning.

Tajkuruna Akter Tithy received the B. Sc. (Engineering) from the department of
Electronics and Communication Engineering (ECE), Hajee Mohammad Danesh Science
and Technology University (HSTU), Dinajpur-5200, Bangladesh in 2019. Currently, she
is a student of M. Sc. (Engineering) in the same department. Her research interest is
performance analysis of communication protocols, OFDM, PAPR.

Md. Mehedi Hasan student of pharmacy, World University of Bangladesh. His Interest
of Deep Learning, Machine Learning and Medical Science.