REALTIME POTHOLE

DETECTION SYSTEM
USING IMPROVED
CNN MODEL - SSD
Internal Guide –
Ms. B. Leelavathy Assistant Professor Vasavi College Of
Engineering

K .Md. Zahid Parvez(1602-20-737-076)

Bathini Ajay Goud(1602-20-737-310)
Abstract
• Roadway maintenance and safety are critical
due to pothole-related risks, including
accidents and increased expenses.
• Traditional pothole detection methods lack real-
time analytics, hindering proactive solutions.
• This study presents a centralized platform for
precise pothole detection using Neo6M GPS
and ESP32 Camera Modules, featuring real-
time mapping, route optimization, and detailed
analytical tools.
• The platform enhances road defect
identification, reduces accident risks, optimizes
travel routes, cuts maintenance costs, and
provide data-driven insights for improved road
conditions and commuter satisfaction.
Precise Problem
Statement
• Real-Time Pothole Detection System is to address the challenge of road damage
detection and reporting. Traditional methods of road inspection are often time-
consuming, require manual effort, and can be subject to human error.
• This project aims to automate the detection of various road damages, such as
potholes, cracks, and obstructions, using advanced Single-Shot Detector (SSD)
technology. The system seeks to provide real-time updates of road conditions to a
backend system, ensuring timely and accurate data for maintenance authorities.
• By integrating with Google Maps, the project also endeavors to enhance route
planning for drivers by providing information on road damages along their
intended path, thereby improving road safety and driving experience. For officials
and maintenance teams, it provides a streamlined workflow for monitoring and
addressing road damages
• Overall, the system aims to improve road safety, minimize vehicle damage and
accidents, and optimize maintenance operations, ultimately contributing to
smarter city management and better public service.
Literature Survey
• Object detection is a vital aspect of computer vision,
employing techniques from traditional methods to
advanced deep learning approaches.
• Deep learning, particularly Convolutional Neural Networks
(CNNs), has revolutionized object detection, offering both
two-stage (e.g., Faster R-CNN) and one-stage detectors
(e.g., SSD and YOLO).
• Single Shot Detector (SSD) stands out among one-stage
detectors for its efficiency and performance. By predicting
detections at multiple scales within a single network using
default boxes, SSD accommodates objects of various sizes
and aspect ratios, ensuring robustness to scale variations.
• SSD excels in balancing speed and accuracy, achieving real-
time object detection at high frame rates without sacrificing
much accuracy. Its simplicity compared to two-stage
detectors simplifies training and deployment, eliminating
the need for complex region proposal networks and
reducing computational costs.
Literature Survey
• Since its introduction, SSD has undergone extensive
study and improvement. Various works have
optimized its architecture, enhanced its
performance, and adapted it to specific applications.
• Additionally, research has explored different aspects
of SSD, such as feature extraction techniques,
anchor box design, and loss function optimization.
SSD’s versatility extends beyond generic object
detection to applications like pedestrian and vehicle
detection, showcasing its adaptability across diverse
computer vision tasks.
• Challenges such as varying lighting conditions,
weather, shadows, and different road surface
textures that can impact detection accuracy were
acknowledged, and strategies to mitigate these
challenges were discussed.
• Integration with Firebase Cloud Firestore for storing
pothole location data, along with Google Maps
integration, allows for real-time updates and efficient
 Data Collection and Preprocessing

Choosing Framework, Architecture, and

Our
Model

Training of Custom Dataset and Initial

Propose
Results

Post-Tuning Strategies

d Work Results and Improvements

A comprehensive website integrated with

Google Maps for analyzing the damages
reported
Block Diagram
and
Architecture
Data Collection and
Preprocessing
• Source: Kaggle
• Explanation: The dataset of 700+ images was
sourced from Kaggle, a platform for machine learning
datasets and competitions.
• Labeling with LabelMe, storing labels in XML
• Explanation: Image annotation was performed using
LabelMe, a popular tool for annotating images with
bounding boxes.
• The annotated data, containing information about the
location of objects in the images, was stored in XML
format.
• This structured data is crucial for training a computer
vision model.
Data Collection and
Preprocessing
• Augmentation of images using Albumentations for dataset
diversification
• Explanation: Albumentations is a powerful Python library for image
augmentation. Image augmentation involves applying various
transformations to the original images to create new training samples,
thereby enhancing the model’s ability to generalize. The augmentation
techniques applied include:
• Random cropping: Extracting random regions from the images to introduce
variability.
• Grayscale conversion: Transforming images to grayscale to account for
different color representations.
• Saturation adjustment: Changing the intensity of colors to handle variations.

• Brightness adjustment: Adjusting the overall brightness of images for

robustness.
• Hue adjustment: Modifying the hue of images to simulate different lighting
conditions.
• Shadow adjustment: Introducing shadows in images to account for diverse
scenarios.
• Night-time simulation: Adjusting images to represent nighttime conditions,
ensuring the model’s adaptability.
• Purpose: Augmentation enhances the model’s performance by exposing
it to a wider range of scenarios, ensuring better generalization and
robustness in real-world conditions.
 Chosen for
flexibility, ease of
use, and strong

Choosing Framework: PyTorch community

support. Enables
seamless
integration with

Framewor
SSD for training
and deployment.

k,
ResNet34 variant
as the backbone
can provide better
Architecture: feature extraction

Architectu
capabilities as
ResNet our project has
large set of
features

re and Used SSD for

predicting the
bounding boxes at

Model
different scales in
a single pass
Model: SSD through a network
rather than two
stage.
Checked with
different
Momentum values
 Training Model Training

• Trained our custom dataset on top

of of SSD with different Momentum,

Optimizer, Learning Rate

Custom Initial Model Evaluation:

Dataset • Presentation of Metrics: Introduced

initial performance metrics,

and Initial showcasing the model’s

achievements.
• Acknowledgment of Challenge:
Results Recognized the challenge of limited
accuracy and set the stage for
improvement.
Results and Analysis

After training with different Hyper Parameters we achieved 88.676 mAP@50

and 0.6764 Train Loss with Momentum 0.1, Learning Rate 0.0005, SGD Loss
Function.
Results and Analysis
Models mAPval@50

YOLOv5m6 80.8

YOLOv5s6 82.2

YOLOv5n6 82.5

YOLOv8n 82.3

YOLOv8s 83.9

YOLOv8m 85.2

YOLOv8l 84.9

YOLOv8x 82.9

Comparison between different object detection Comparative Analysis

models with SSD
0.1 0.001 ADAM 25.769 0.545 0.9 0.01 SGD 85.85 0.976 0.9 0.001 LR ADAM 49.9 1.892
Momentu Optimize mAP Train Moment Optimiz mAP Train Momentu Optimize mAP Loss
m r Loss u er Loss r Function
0.1 0.0005 SGD 88.676 0.6764 0.5 0.0000 SGD 13.15 mAP 39.944
Momentu LR Optimizer mAP Train Loss Momentu 002 Optimizer Train Loss
m m LR
Results
Results
 The conclusion of the real-time pothole detection
project highlights the significance of parameter
tuning in achieving optimal model performance, with
findings indicating a momentum of 0.1 and a

Conclusi
learning rate of 0.005 as optimal configurations for
the highest mean Average Precision (mAP@50) score
of 88.676. The implications extend to infrastructure
monitoring, road maintenance, and safety

on and enhancement, with the system facilitating proactive

maintenance, enhancing road safety, and supporting
data-driven decision-making in urban planning.

Future Future research avenues include exploring advanced

model architectures like ensemble learning and
multi-modal data fusion, leveraging IoT devices and

Scope
edge computing for real-time processing, and
developing intuitive user interfaces and visualization
tools to enhance community participation.
Collaboration with academic institutions, industry
stakeholders, and governmental agencies is
essential for accelerating advancements in
automated infrastructure monitoring solutions.
Software and Hardware
Requirements
Software Requirements
Python 3.10+
OpenCV
PyTorch
Albumentation
LabelMe
SSD 
Google Colab
Javascript enabled Web Browser
Hard Requirements
Physical Camera with IP Streaming (ESP32)
Neo6M GPS Module
System with Nvidia GPU (If Google Colab is not used)
THANKYOU
Team B13