REAL-TIME POTHOLE DETECTION SYSTEM USING

IMPROVED CNN MODEL – SSD

A
Project Report
Submitted in partial fulfilment of the
Requirements for the award of the Degree of

BACHELOR OF ENGINEERING
IN
INFORMATION TECHNOLOGY
By

KHATEEB MOHAMMED ZAHID PARVEZ - 1602-20-737-076

BATHINI AJAY GOUD - 1602-20-737-310

Under the guidance of

Ms. B. Leelavathy

Assistant Professor

Department of Information Technology

Vasavi College of Engineering (Autonomous)
ACCREDITED BY NAAC WITH 'A++' GRADE
(Affiliated to Osmania University)
Ibrahimbagh, Hyderabad-31
2024
 Vasavi College of Engineering (Autonomous)
ACCREDITED BY NAAC WITH 'A++' GRADE
(Affiliated to Osmania University)
Hyderabad-500 031
Department of Information Technology

DECLARATION BY THE CANDIDATE

We, Khateeb Mohammed Zahid Parvez and Bathini Ajay

Goud bearing hall ticket number, 1602-20-737-076 and 1602-20-737-
310 hereby declare that the project report entitled Real-time project
pothole detection system on vehicle using improved CNN model-
SSD under the guidance of Ms. B. Leelavathy, Assistant Professor
Department of Information Technology, Vasavi College of
Engineering, Hyderabad, is submitted in partial fulfilment of the
requirement for the award of the degree of Bachelor of Engineering
in Information Technology

This is a record of bonafide work carried out by us and the

results embodied in this project report have not been submitted to any
other university or institute for the award of any other degree or
diploma.

Khateeb Mohammed Zahid Parvez

1602-20-737-076

Bathini Ajay Goud

1602-20-737-310
 Vasavi College of Engineering (Autonomous)
ACCREDITED BY NAAC WITH 'A++' GRADE
(Affiliated to Osmania University)
Hyderabad-500 031
Department of Information Technology

BONAFIDE CERTIFICATE

This is to certify that the project entitled Real-time pothole detection

system using improved CNN models being submitted by Khateeb
Mohammed Zahid Parvez and Bathini Ajay Goud bearing 1602-20-
737-076 and 1602-20-737-310 in partial fulfilment of the requirements
for the award of the degree of Bachelor of Engineering in Information
Technology is a record of bonafide work carried out by them under my
guidance.

Ms. B. Leelavathy Dr. K. Ram Mohan

Rao,
Assistant Professor Professor,
Internal Guide HOD, IT
 ACKNOWLEDGEMENT
The satisfaction that accompanies that the successful
completion of the project would not have been possible without the
kind support and help of many individuals. We would like to extend
our sincere thanks to all of them.

It is with immense pleasure that we would like to take the

opportunity to express our humble gratitude to Ms. B. Leelavathy,
Assistant Professor, Information under whom we executed this
project. We are also grateful to Dr. S. Sree Lakshmi, Assistant
Professor (Sr. Scale), Information Technology for her guidance. Their
constant guidance and willingness to share their vast knowledge made
us understand this project and its manifestations in great depths and
helped us to complete the assigned tasks.

We are very much thankful to Dr. K. Ram Mohan Rao, Pr,

Information Technology, for his kind support and for providing
necessary facilities to carry out the work.

We wish to convey our special thanks to Dr. S. V. Ramana,

Principal of Vasavi College of Engineering for giving the required
information in doing my project work. Not to forget, we thank all
other faculty and non-teaching staff, and my friends who had directly
or indirectly helped and supported me in completing my project in
time.

We also express our sincere thanks to the Management for

providing excellent facilities. Finally, we wish to convey our gratitude
to our family who fostered all the requirements and facilities we need.
 Abstract

In the dynamic landscape of computer vision, object detection serves as a

cornerstone task empowering machines to identify and localize multiple objects
within images. This project embarks on the development of an object detection
system utilizing the Single Shot Detector (SSD) framework, celebrated for its
efficiency, speed, and accuracy.

SSD operates by generating fixed sets of default bounding boxes (anchors)

across multiple feature maps and directly predicts class scores and bounding box
offsets for each default box. This unique architecture enables SSD to efficiently
capture objects at various scales within a single forward pass, distinguishing it
from two-step detectors relying on region proposal networks.

The project's primary aim is to harness SSD's capabilities to detect and

classify road damages, particularly emphasizing scenarios necessitating precision
and real-time processing. To accomplish this, the project integrates a pre-trained
ResNet34 backbone, renowned for its depth and performance in image recognition
tasks.

Furthermore, the system extends its functionality by seamlessly integrating

the ESP32 camera module and the Neo6M GPS Module, enabling real-time video
capture and precise geo-location tracking. Additionally, integration with Google
Maps facilitates dynamic assessment of detected objects' spatial context,
enhancing situational awareness and decision-making.

Beyond mere detection, the system furnishes insights into object bounding
box coordinates and associated labels, fostering a comprehensive understanding of
detected entities' spatial context and semantic relevance within the image.

By harnessing the power of modern deep learning frameworks such as

PyTorch and leveraging meticulously curated datasets, this project endeavors to
push the boundaries of object detection, unlocking applications across diverse
domains ranging from autonomous driving to surveillance and beyond.
List of Figures vii
List of Tables viii
List Of Abbreviations xi
1 INTRODUCTION 1
1.1 Problem Statement – Overview 1
1.2 Proposed Work 1
1.3 Scope & Objectives of the Proposed Work 2
1.4 Organization of the Report 3
2 LITERATURE SURVEY 5
3 PROPOSED WORK 7
3.1 Block Diagram 7
3.2 Algorithm 9
4 EXPERIMENTAL STUDY 14
4.1 Data Sets 14
4.2 Software Requirements 15
4.3 Hardware Requirements 16
4.4 Preprocessing 18
4.5 Parameter Setting 21
4.6 Results 24
4.7 Analysis 27
5 CONCLUSION AND FUTURE SCOPE 32
REFERENCES 34
 LIST OF FIGURES

Fig. No Description Page No

i Block diagram of proposed work 7

ii Use Case Diagram 8

iii Dataset after applying Augmentation Techniques 15

iv Neo6M GPS Module interfacing with ESP32 16

v Neo6M GPS Module 17

vi ESP32 Camera Module 17

vii SSD Model Architecture 22

viii Detection of Potholes in Different Lighting Conditions by our 24

SSD model
xi Detection of Potholes of Different Sizes, Shapes, and 25
Dimensions
x Detection of False Objects like Trees, Shadows, Vehicles, 25
Healthy Roads
xi Comprehensive website integrated with Google Maps API for 26
analyzing results and viewing potholes in a route

xii Iterations vs Train Loss and Epoch vs mAP@50 28

xiii Iterations vs Train Loss and Epoch vs mAP@50 29

xiv Iterations vs Train Loss and Epoch vs mAP@50 29

xv Iterations vs Train Loss and Epoch vs mAP@50 30

xvi Iterations vs Train Loss and Epoch vs mAP@50 30

xvii Iterations vs Train Loss and Epoch vs mAP@50 30

xviii Iterations vs Train Loss and Epoch vs mAP@50 31

 LIST OF TABLES

Table No. Table Name Page No

I Results obtained while training the SSD model 27
 LIST OF ABBREVIATIONS

1. SSD – Single Shot Detector

2. YOLO – You Only Look Once
3. SGD - Stochastic Gradient Descent
4. ADAM – Adaptive Movement Estimation
5. CNN – Convolution Neural Networks
6. mAP – Mean Average Precision