Improving Road Safety with Pothole Detection in

Autonomous Vehicles
John Carlo L. Astoveza, Joel Jr. T. Almerol, John Vincent M. Cabugnason
Information Technology Department
José Rizal University
Mandaluyong City, Philippines
[email protected], [email protected], [Insert email]
Abstract: Deep learning could be used to create a model that can
accurately identify different objects on the road, such as
This paper uses the Fashion MNIST dataset as a case
pedestrians, vehicles, traffic signs, and so on. This is critical
study to demonstrate the application of Multinomial Logistic
Regression (MLogReg) to the image classification problem. for self-driving cars to navigate safely and avoid collisions.
MLogReg is a statistical technique that is assessed for its Convolutional neural networks (CNNs) are commonly used
capacity to classify images according to multiple predictors. It to solve this problem because they can extract features from
can handle categorical outcomes with more than two classes. In images and learn to classify them. However, not all CNNs
addition, the paper investigates the use of neural networks in are equal in this regard. Some may be too general and fail to
conjunction with MLogReg, leveraging their ability to handle capture the specific characteristics of the objects, while
intricate data and patterns. Using the Fashion MNIST dataset, others may be overly complex and necessitate a large
the model is trained and tested as part of this methodology, and
amount of computational power. As a result, we propose
various hyperparameter settings are thoroughly examined. The
developing a specialized CNN-based model for single-
results display the various models' performances, with an
emphasis on the top-performing model, and provide object image classification in the domain of autonomous
recommendations for further research and development. vehicles.

Keywords: Pothole detection is an important task for autonomous

cars since it can improve road safety and prevent damage.
Multinomial Logistic Regression, Neural Networks, Image CNNs are ideal for this task because they can learn from
Classification, Fashion MNIST, Hyperparameter Tuning, Model
local details in road photos. Recently, a new method based
Performance, Stochastic Multinomial Logarithmic Model,
Khatri-Rao Decomposition, Kronecker Factorization. on location-aware CNNs was proposed that obtained
excellent accuracy and surpassed earlier methods. CNNs can
I. INTRODUCTION also be used to classify images in general since they
resemble the organic structure of the visual cortex. They
Convolutional Neural Networks (CNNs) are a type feature a straightforward yet effective architecture that
of neural network that excels at visual data processing and makes them simple to comprehend and apply [2].
tasks like image classification and object detection. They are
made up of several layers that perform various mathematical II. RESEARCH OBJECTIVES
operations on the input data, such as convolution, pooling,
and activation. These layers extract features from images Autonomous vehicles (AVs) are a new technology that
and train themselves to recognize patterns relevant to the has the potential to change the transportation industry. AVs
task at hand [1]. are capable of operating autonomously, communicating with
other agents, and making data-driven decisions. However,
CNNs are inspired by the structure and function of the AVs confront other challenges, including safety concerns,
visual cortex in the brain, which consists of neurons that legislative impediments, and ethical quandaries [3].
respond to different stimuli in the visual field. By
mimicking this biological mechanism, CNNs are able to A specific CNN-based model for single-object picture
achieve remarkable results in various domains of computer classification could be useful for autonomous vehicles,
vision and artificial intelligence. One of the most influential particularly in detecting road potholes [3]. To create such a
papers that introduced CNNs to the world was “Gradient- model, we’ll need to collect data that replicates the real-
Based Learning Applied to Document Recognition” by Yann world circumstances and obstacles that autonomous vehicles
LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. experience. We will also need to examine the domain and
This paper presented a CNN model called LeNet-5 that comprehend the aspects and challenges of pothole
could recognize handwritten digits with high accuracy [1]. recognition, such as changing lighting conditions,
occlusions, and different types of road surfaces. We will be
able to construct and train a model that can accurately and
reliably classify potholes in various settings as a result of potholes, and helps to improve road safety. On public
this. datasets, the research compares the method to other deep
learning approaches and demonstrates its accuracy. The
III. LITERATURE REVIEW report also examines the history of road surface monitoring
The literature has given a CNN model strategy for tools such as vibration, computer vision, and 3D
single-object picture classification that can improve object reconstruction methodologies. It goes over the consequences
detection in autonomous vehicles, particularly for potholes of poor road conditions on driving comfort and
[4]. To detect and track objects in real-time, the method transportation safety, as well as the possibility for vehicle
employs deep learning techniques such as the YOLOv4 and damage. Furthermore, the article emphasizes the economic
Deep SORT algorithms. The method also includes a significance of strong infrastructure, which supports
probabilistic attention mechanism with transformers, which economic growth and worldwide trade while facing
assists the model in focusing on the most essential part of an increasing pressures from reconstruction and maintenance
image, improving detection efficiency and accuracy [5]. The demands.
method has been tested on publicly available datasets and IV. Methodology
shows comparable accuracy to other cutting-edge deep
learning methods [6]. The results show that this method can Images with only one object of interest, such as a
significantly improve object detection in autonomous tumor, a product, or a person, are known as single-
vehicles, including potholes, and hence promote safe object images. These images can be used in a variety
navigation. of applications, including medical imaging, retail, and
Roads and highways in poor condition affect traffic, surveillance, where the goal is to detect, classify, or
cause accidents, and impede the growth of the economy. segment the object of interest. However, relevant
Modern societies cannot succeed economically or socially datasets are required to train and evaluate models for
without a well-built and maintained roadway system. In [7], single-object image tasks. These datasets should have
The detection of irregularities on the road surface has been high-quality annotations, a wide range of scenarios,
explored for a long time. Over time, the techniques for and realistic scenarios. For example, we obtained a
identifying irregularities in the road surface and potholes suitable dataset for pothole detection from Kaggle,
have evolved. Three methods have been used historically to
which included 329 images of potholed roads and 352
monitor road surfaces: vibration-based, computer vision-
based, and 3D reconstruction. A 3D laser scanner is needed
images of normal roads.
for the 3D reconstruction method in order to accurately Our project aimed to create self-driving vehicles
capture the current condition of the road surface through
that can detect potholes in the road. To accomplish
surface scanning.
this, we gathered and annotated a dataset of images
In order to give commuters good road infrastructure, containing various objects that could be useful for
road surface monitoring is a major issue. According to Singh surveillance or detection. Depending on whether or not
et al., 2017, road potholes and bumps impact driving the image showed a pothole, we labeled it as Normal
comfort and transportation safety, so spotting them is or Pothole.
essential to road condition monitoring. The vehicle itself
sustains damage to its suspension, misaligned steering, and To create a Convolutional Neural Network (CNN)
punctured tires which also lead to accidents. model for our problem, we considered factors such as
image dimensions and the number of classes. We made
Good roads and infrastructure are essential for
economic development because they allow people and certain that all of the images were the same size and
things to move around more easily and create connections that the classes were evenly distributed across the
that promote global trade [8]. On the other hand, pressure is dataset. We avoided any potential bias or distortion in
mounting on the road infrastructure. On the one hand, the CNN performance this way.
numerous roads and their components need to be rebuilt or
maintained because a significant portion of the One of the most important steps in developing
transportation infrastructure is antiquated. high-performance CNNs is fine-tuning the
hyperparameters, which include the number of folds,
This paper describes a CNN-based approach for learning rate, batch size, and others. This means we
detecting potholes in self-driving scenarios. The technology
train the model on a subset of the dataset and then
enhances object identification performance, particularly for
fine-tune the parameters based on how well it performs
on the validation set. Hyperparameter tuning helps us Sciences and Techniques of Automatic Control and Computer
Engineering (STA), 2022, pp. 89–94.
to increase both the accuracy and the efficiency of our [7] A. K. Pandey, R. Iqbal, T. Maniak, C. Karyotis, S.
model. Akuma, and V. Palade, “Convolution neural networks for
pothole detection of critical road infrastructure,” Comput.
Annotated datasets are used for training and Electr. Eng., vol. 99, no. 107725, p. 107725, 2022.
[8] S. Gupta, P. Sharma, D. Sharma, V. Gupta, and N.
validation of a CNN model, allowing the model to Sambyal, “Detection and localization of potholes in thermal
learn object identification and categorization through images using deep neural networks,” Multimed. Tools Appl.,
backpropagation. K-fold methods are used to ensure vol. 79, no. 35–36, pp. 26265–26284, 2020.
[9] A. Kumar, “Pothole Detection Dataset.” 30-Dec-
robust generalization by dividing the data into different 2019.
training and validation sets for each fold. The model's [10] A. Germanov, “How to detect objects in images
using the YOLOv8 neural network,” freecodecamp.org, 04-
performance is evaluated after training on separate May-2023. [Online]. Available:
validation datasets that were not used during training, https://www.freecodecamp.org/news/how-to-detect-objects-in-
allowing adjustments based on validation results. This images-using-yolov8/. [Accessed: 22-Nov-2023].
[11] H. Salaudeen and E. Çelebi, “Pothole detection using
iterative K-fold approach improves the model's image enhancement GAN and object detection network,”
dependability and adaptability to a wide range of Electronics (Basel), vol. 11, no. 12, p. 1882, 2022.
datasets, ensuring its effectiveness in real-world
applications.
Object detection, a computer vision task that seeks
to locate and identify things in images or videos, has a
special application in pothole detection. Numerous
fields, including robotics, autonomous driving, and
video surveillance, depend on object detection [10].
We report a new approach to pothole detection in this
research that blends a super-resolution network with an
object detection network. Our approach can reliably
identify potholes from photos, even in difficult
situations like far-off or tiny potholes, poor image
quality, and complicated backdrops. We show that our
approach performs more accurately and robustly than
the pothole detection research that is currently
available [11].
V. REFERENCES
[1] K. O’Shea and R. Nash, “An Introduction to
Convolutional Neural Networks,” arXiv [cs.NE], 2015.
[2] H. Chen, M. Yao, and Q. Gu, “Pothole detection
using location-aware convolutional neural networks,” Int. J.
Mach. Learn. Cybern., vol. 11, no. 4, pp. 899–911, 2020.
[3] S. A. Bagloee, M. Tavana, M. Asadi, and T. Oliver,
“Autonomous vehicles: challenges, opportunities, and future
implications for transportation policies,” J. Mod. Transp., vol.
24, no. 4, pp. 284–303, 2016.
[4] R.-A. Bratulescu, R.-I. Vatasoiu, G. Sucic, S.-A.
Mitroi, M.-C. Vochin, and M.-A. Sachian, “Object detection in
autonomous vehicles,” in 2022 25th International Symposium
on Wireless Personal Multimedia Communications (WPMC),
2022.
[5] A. Gupta, K. Illanko, and X. Fernando, “Object
detection for connected and autonomous vehicles using CNN
with attention mechanism,” in 2022 IEEE 95th Vehicular
Technology Conference: (VTC2022-Spring), 2022, pp. 1–6.
[6] A. Albouchi, M. A. Hajjaji, and A. Mtibaa, “Deep
learning-based object detection approach for autonomous
vehicles,” in 2022 IEEE 21st international Ccnference on