Detection and Classification

using Deep Models

Presented
By
Sourajit Maity
Outline

• Introduction
• Challenges
• Research Gaps
• Research Works
• Research Ongoing
• Conclusion
• Publications
 Introduction

Automatic vehicle detection and classification (AVDC)

systems have become important for real-time traffic
monitoring and management.

AVDC requires collection of suitable data with real time

traffic information, and automatic vehicle classification and
detection methodologies.

Fig 1. Vehicle Detection

Research Challenges

Making models that

Image and video are robust enough
datasets have its to use for inputs
own challenges. taken in different
weather conditions.

Models dealing
with data in Due to congested
daylight conditions traffic, a single
might not work for frame has multiple
data taken in night vehicles.
conditions.
Research Gaps

• Many datasets used over the years give almost 100% accuracy.
• Huge samples are needed to train deep learning based models.
• Many datasets are not accurately processed and annotated .
• In Indian sub-continent road conditions, vehicle types, traffic
scenarios are dissimilar than those found in developed
countries.
• Hence, most of the research articles available in the literature
may not be suitable for such cases.
• Some datasets have very few classes.
 • The current survey encompasses all key research papers published
on AVD between the year 2010 and 2020.
• It gives a general overview of the several approaches applied for
both localizing as well as recognizing different vehicle classes
Last Decade in using both machine learning and deep learning approaches.

Vehicle Detection • It includes the most widely used AVD datasets (comprising of
video as well as still-image datasets) used for both vehicle
and localization as well as classification problems.

Classification: A • It also presents a critical review of the deep learning based methods, which is the
current trend, used for AVD problem.
Comprehensive • Finally, it analyses different prospects of future research achievements in this
field
Survey
• Published in Archives of Computational Methods in Engineering -Springer
 • To the best of our knowledge, this paper discusses most
of the research papers on VMMR published between the
Two decades of year 2004 and 2023.
• We have identified as well as provided a general
vehicle make and overview of various methodologies for VMMR based on
model both machine learning and deep learning models
supported by some in-depth discussion.
recognition –
Survey, • We have provided information regarding several vehicle related
datasets used for VMMR task.

challenges and • Finally, we have presented various research gaps, and their potential
solutions for the said field’s research advancements in the future.
future directions
• Published in Journal of King Saud University - Computer and
Information Sciences - Sciencedirect -Elsevier
Current Datasets and Their Inherent Challenges for
Automatic Vehicle Classification
• Also, we have given a comparative study of
the different types of datasets used for
classification along with their pros and cons.
• This study presents a comprehensive survey
of the datasets available for AVC and vehicle
model and make recognition (VMMR)
published in the last 10 years highlighting
their inherent challenges.

• Published in Machine Learning for Cyber

Physical System: Advances and Challenges-
Springer
Performance Comparison of Various
YOLO Models for Vehicle Detection:
An Experimental Study

• In this paper, we focus on three major object

detection algorithms under the YOLO family,
namely YOLOv5, YOLOv7, and YOLOv8 for the
purpose of vehicle detection,
• Discuss the architectural differences of these
variants.
• Performance comparison of these models, and
in doing so, we use two recently introduced AVD
datasets developed for the Indian subcontinent,
namely JUVDsi v1 and IRUVD.

• Published in Proceedings of Data Analytics and

Management- ICDAM 2023, Volume 3 Springer
JUVDsi v1: developing and benchmarking a new
still
image database in Indian scenario for automatic
vehicle detection

• The image database is properly annotated to measure

the performance of any algorithm developed for
automatic localization and classification of vehicles in
an unconstrained environment.
• Different complexities are added to the images to make
it challenging as well as realistic.
• Nine different classes of vehicles are presented in this
database – very few of the existing databases have
considered such a variety of vehicle classes.
• Three models namely, You Only Look Once (YOLO) ,
Region-Based Convolutional Neural Networks (R-CNN) ,
and Region-Based Fully Convolutional Networks (RFCN)
are used. Finally an ensemble method, called Weighted
Boxes Fusion (WBF) is implemented.
• Published in Multimedia Tools and Applications,
Springer
JUIVCDv1: development of a still-
image based dataset
for indian vehicle classification

• This dataset offers a realistic image representation of the

traffic situation in India, which isbvery different from that of
other developed countries. Vehicle images captured in
variousbscenarios are considered. A total number of 6335
vehicle images can be found in this dataset..
• Researchers may take this dataset to evaluate the
effectiveness of their methods for autonomous vehicle
localization and categorization.
• The vehicle images in the collection are taken in different
weather conditions. Therefore, the model is resilient
enough to handle data collected in a variety of
meteorological scenarios.
• We have benchmarked this dataset using an MVE classifier
combination approach which achieves 95% accuracy.
• Published in Multimedia Tools and Applications, Springer
XMR_Net: A Deep Model for Vehicle Make
and Model Recognition using Still-images

• In this work, we have proposed an ensemble of

attention-aided three deep CNN models, called
XMR_Net, for VMMR.

• In this paper, initially, we have used five standard

convolutional neural network (CNN) models, namely
Inceptionv3, Xception, InceptionResNetv2,
MobileNetV2, and ResNet152v2 for VMMR.

• We have also used an attention mechanism to these

models. To increase accuracy of the overall model

• we have chosen three best base learners from these

five CNN models, and formed an ensemble model. The
final model is called XMR_Net
SimSANet: A Simple Sequential Attention aided Deep Neural Network
for Vehicle Make and Model Recognition

• We present Simple Sequential Attention Network (SimSANet),

a multikernel-based sequential attention-based model, which
efficiently extracts the most discriminative information by
combining both global and local features.
• It also offers significant advantages in speed, effectiveness,
and efficiency, requiring far fewer parameters compared to
existing models.
• To demonstrate the significance of each layer in the
architecture of the suggested model, along with the
recommended values for the hyperparameters, we perform a
statistical analysis and ablation studies. We utilize the Grad-
CAM to show the efficacy of the proposed model.
• We conduct extensive experiments on multiple public VMMR
benchmark datasets to ensure the effectiveness of the
proposed model.
SEFFNet: Snapshot Ensemble-based Feature Fusion Network for Skin Cancer
Classification

• We propose a novel snapshot ensemble-based feature

fusion network for skin cancer classification..
• We introduce a hybrid loss function, which dynamically
addresses overfitting and class imbalance issues.
• Our model achieves superior performance without the
need for pre-processing or data augmentation
techniques
• The proposed model is evaluated on two publicly
available standards skin lesion datasets.
A Feature Fusion based Custom Deep Learning
Model for Vehicle Make and Model Recognition
• Deep Feature Fusion: It fuses two feature maps
that are extracted from input images in two parallel
paths containing two different base models.
• Modified CBAM: Taking inspiration from the
original CBAM attention [14], a modified attention
mechanism is used, which extracts channel and
spatial information in parallel and combines them
with input feature maps.
• Generalized Approach: Our model is evaluated on
two VMMR datasets, namely Stanford Cars and
Comp-CarsSV, and achieved 93.51% and 99.03%
test accuracies, respectively.
MIXNet: A Deep Model for Cancer Type
Identification from CT-Scan Images
• In this study, we have introduced a deep
learning model called MIXNet, which
comprises an ensemble of attention-based
CNN models.
• Three CNN models,namely, MobileNetV2 ,
InceptionResNetV2 and Xception have been
used. Additionally, an attention layer has been
added to the models, followed by the
application of a majority voting ensemble
approach to achieve better results.
• To appraise the model’s performance, we
have considered the LC25000 and the Chest
CT-scan images dataset.
Generative AI Synthesis and
Linguistic Comprehension

• Our primary objective revolves around

automating traditionally human-intensive tasks,
such as code debugging and legal procedure
summarization, with the overarching aim of
bolstering productivity and operational efficiency.
• To achieve this, we harness the capabilities of a
transformer-based model, specifically leveraging
a Generative Pre-trained Transformer
architecture.
• Developing user-friendly interfaces and APIs to
facilitate seamless interaction between users
and the AI assistant.
DigiDerma: An
Application for
Skin Disease
Prediction
using Attention
enabled Deep • We have developed a mobile application named DigiDerma for skin disease
prediction from input images of human skin.

Transfer • To achieve a light weight framework for the proposed DigiDerma system, a
transfer learning based MobileNetV2 model is used. The system is aided by a

Learning model Squeeze-and-Excitation attention network for enhancing the prediction

performance.
• The proposed system is trained and evaluated on HAM10000 dataset. Also,
the performance of the proposed DigiDerma system is compared with some
recent methods.
HARVEST-HUB: IOT
BASED SMART
AGRICULTURE SYSTEM

• The system promotes the regularity of watering

the plants in an efficient manner.
• This automated system provides a user-friendly
interface to the users where they can easily get
the current records for all the necessary
requirements for plant nurturing thereby aiding
the farmers to reap benefits in using this
automated system to reduce the over-wastage of
resources and to monitor all the activities
without being physically present in the sight.
• This smart automation system can have the
capability to fetch the real-time data from the
sensors and can produce huge efficiency and
accuracy over that.
Conferences

• Attended 4th ICDAM (International

Conference On Data Analytics and
Management) 2023 and presented my
research paper on Performance Comparison
of various Yolo models for Vehicle
Detection: AnExperimental Study.
• Attended the Global Conference on
Applications of Artificial Intelligence and
Data Science (AAIDS 2024) and presented
my research paper on XMR_Net: A Deep
Model for Vehicle Make and Model
Recognition using
Research
Ongoing

• Developing vehicle detection

dataset for adverse weather
condition.
• Developing a deep learning based
vehicle detection lightweight
model.
• Developing a deep learning based
monkeypox classification model.
Research
Ongoing

• Developing Generative AI based LLM

model for human-intensive tasks namely
NexZenith: An AI Software As Service
Solution
• Working on feature selection-based vehicle
classification using genetic algorithm.
• Working on survey article on vehicle
detection methods.
• Writing a book chapter on available
datasets on vehicle detection.
Conclusion

MAIN AIM IS TO MAKE AN EFFICIENT AVDC SYSTEMS SHOULD BE WORKING EFFICIENTLY ON DATASETS WILL BE DEVELOPED FOCUSING ON
SYSTEM THAT CAN BE USED TO SOLVE THE IMAGES AND VIDEOS TAKEN IN DIFFERENT INDIAN TRAFFIC AND ROAD CONDITIONS.
AMBIGUITY ISSUES DUE TO OVERLAPPING WEATHER CONDITIONS.
VEHICLES.
 A. Bhattacharyya, A. Bhattacharya, S.
S. Maity, A. Bhattacharyya, P. K. Singh, M.
Maity, P. K. Singh, R. Sarkar, “JUVDsi v1:
Jindal, R. Sarkar, “Last Decade in Vehicle
Developing and Benchmarking a New Still
Detection and Classification: A
Image Database in Indian Scenario for
Comprehensive Survey”, Archives of
Automatic Vehicle Detection”, Multimedia
Computational Methods in Engineering,
Tools and Applications, Springer, 2023,
Springer, 2022, Impact Factor: 7.302
Impact Factor: 2.757

S. Maity, A. Chakraborty, P. K. Singh, and

Publications:
S. Gayen, S. Maity, P. K. Singh, Z. W. Geem,
R. Sarkar, “Performance Comparison of
and R. Sarkar, “Two Decades of Vehicle
Various YOLO Models for Vehicle
Make and Model Recognition–Survey,
Detection: An Experimental Study,” in
Challenges and Future Directions,” Journal
International Conference on Data
of King Saud University-Computer and
Analytics & Management, Springer, 2023,
Information Sciences, p. 101885, 2023.
pp. 677–684.

Maity, S., Singh, P. K., Kaplun, D., & Sarkar,

Maity, Sourajit, Debam Saha, Pawan R. (2024). Current Datasets and Their
Kumar Singh, and Ram Sarkar. "JUIVCDv1: Inherent Challenges for Automatic Vehicle
development of a still-image based Classification. In Machine Learning for
dataset for indian vehicle Cyber Physical System: Advances and
classification." Multimedia Tools and Challenges (pp. 377-406). Cham: Springer
Applications (2024): 1-28. Nature Switzerland.1 0.66328125
0.67265625 0.6421875 0.48515625
Thank You