PAPER CONTEST - 2020 28th March 2020

DETECTION OF ROAD CRACKS AND

POTHOLES USING IOT DEVICE
Virushini R#, Varsha E*, Kirthicka S#
#
First-Third Department, First-Third University
Address
1
[email protected]
3
[email protected]
Second Company
*

Address Including Country Name

2
[email protected]

R.M.K College of Engineering and Technology, Chennai

Page1
 PAPER CONTEST - 2020 28th March 2020
Abstract—A pothole is a special case of road distress. It In this paper they have proposed a simple robust design
for an affordable device which suits the drivers in this
presents a grave danger to human life. The occurrence of
country. This proposal says that there is no need of using
pothole has increased rapidly in extraordinary weather such as
smart phones for sensing of reports. The system can be
heavy rain, snowfall, and has a great influence on traffic safety
and road damage. In this paper we propose a simple affordable
installed in the vehicle itself that detects the potholes using
device that will be suitable for the drivers. Based on the image processing of Rasberry-Pi microcontroller. [7]The
structure of the digital image processing and the structural need for road monitoring road is not that much easy .There
characteristics of the design engineering, this pre analyses the are many different methods and approaches for control of
practical application value of the digital image processing quality of roads .In this article they have introduced light
technology in the crack identification of machine tools weight architecture for analyzing potholes with the data
structures and established a bridge structure health monitoring collected with smartphones.
system based on digital image processing technology. This In this model some new algorithms are added for real time
system can be established in a movable vehicle with an IOT
road anomaly detection using smart phones.[8]A real time
device. The images of the pothole is taken and processed using
an algorithm with the help of MATLAB. And then images are watershed algorithm has been proposed for more number of
collected and with the help of global positioning system, the potholes on asphalt pavement. The algorithm uses inverted
location and the images of the damaged road are registered as a binary using the technique otsu thresholding to identify the
complaint to the highway department automatically in order to optimal threshold value of an image in an inverted color
improve the road condition. space. The next technique used is morphological technique
with open and close kernels for filtering small noises and
Keywords— pothole, road damage, IOT, Global Positioning bold potholes edges on the image .The last method is using
System, MATLAB. the distance transform for identifying the markers on pre-
water shed phase image before applying watershed
I. INTRODUCTION algorithm. [9]
Generally road surface has irregularity and they are A pothole is one of dangerous threat to the drivers as well.
classified as cracks potholes ditches surface defects. [1]The Because of this potholes more accidents occurs due to
pothole is a type of failure on an asphalt pavement. The sudden steering of the vehicle wheel and making hard
potholes have 2 visual properties .First, potholes have areas turning of vehicles by late detection .While driving it is
with low intensity and they are darker than nearby difficult to find where the exact potholes are present. As
pavement. Second the textures present in potholes are number of pavement has been increased, pothole detecting
coarser. But these two visual properties are not enough for has become a great challenge in modern society. Using
detecting potholes accurately as there is lack of texture. [2] sensors, several methods suggest pothole detecting.
All people are in need of road with good condition as the Meanwhile these methods needs an installation in vehicle for
activities of people are indirectly depending on the road collecting data of the pavement. In this paper performance in
conditions. To keep roads in better condition pothole detecting potholes have been investigated with classification
detection must be done. Pothole detection plays an important of image based on deep convolution neural network models.
role because even small damages on the road is not [10] The basic components of land transportation system are
immediately handled, then it might become a serious issue asphalt roads and the quality of asphalt roads will decrease
so that danger gets increased. And cost for repairing will be during the use stage because of deterioration of road surface.
greater. Road damage detection cannot be done manually. Some road pavement distress may appear on road surface
So the supporting technology used to detect road damage is such as potholes and cracks. For improving the efficiency of
computer vision.[3] The pothole is one of the important road pavement inspection, some new methods of remote
factors for drivers. It causes accident, decrease of sensing data are widely to detect pavement distress. Here,
concentration of driver. The pothole detection method has 3 the multispectral pavement were used to distinguish between
types such as 3-axis acceleration sensor, camera sensor and normal pavement and pavement damages using machine
laser sensor. These 3 types have advantages and algorithms such as support vector machines ,artificial neural
disadvantages. Here in this paper camera sensor is used. [4] network and random forest.
Two novel feature extractions are been proposed based on
II. METHODOLOGY
regional and Cartesian profiling of orthogonal axes. This
feature worked well with pothole based problems. It also has This part discusses the following: A. Proposed System,
benefit of decoupling feature extraction. The orthogonal B..Detection of Potholes, C.Reporting to the server, D.
axes features also works well with decision tree, and System Integration
regional profiling works well with support vector machine.
[5] One of the causes of road accidents occurs in developing A. Proposed System
countries such as Philippines is mainly because of road In this project, we present a robust pothole detection
damages such as potholes. At the same time maintenance of algorithm that is both accurate and computationally efficient.
local roads are not done, so the checking of the potholes are A disparity map is first transformed to better distinguish
done manually. between damaged and undamaged road areas. To achieve

R.M.K College of Engineering and Technology, Chennai

Page2
 PAPER CONTEST - 2020 28th March 2020
greater disparity transformation efficiency, golden section
search and dynamic programming are exploited to estimate
the transformation parameters. The Otsu’s thresholding
method is then used to extract potential undamaged road
areas from the transformed disparity map. The dispersion in
the extracted areas are modelled by a quadratic surface using
least square fitting.
To improve disparity map modelling robustness, the surface
normal is also integrated into the surface modelling process.
Furthermore, random sample consensus is utilized to reduce
the effects caused by outliers. By comparing the difference
between the actual and modelled disparity maps, the
potholes can be detected accurately. Finally, the point clouds
of the detected potholes are extracted from the reconstructed
road surface. The successful detection accuracy of the
proposed system is approximately 99.6%. The main aim of
this project to be monitoring each and every crack and
potholes in road based on image processing updating to the
webpage via IOT to the Highway Department.

B. Detection of Potholes
The detection system of potholes is installed in the vehicle
with proper dimensions. The location of the camera plays an The acquired input image is processed be the block
important role for detection of potholes. The camera module diagram. In Pre processing we use Median filter and image
is located at the top of the car and placed securely. The enhancement. Median filter is a technique used for removing
module is kept behind the rear mirror so that reliable and the noise from the image. This noise filtering is done in the
accurate results can be obtained. It is important that the preprocessing step to improve the results in the later stages.
camera should be stable and secured as much as possible The main idea for using this filtering process is to run
since, changes in the position could cause the potholes through frame pixel by pixel and separating each pixel from
undetected. When the vehicle is in motion the camera the neighbouring pixel. This helps to avoid the processing of
detects the road continuously senses the road and takes the boundaries with or without cropping the image boundaries.
frame for each second. And process the image using the In the gray scale analysis the method called Columnar mean
digital image processing. is used for obtaining the gray scale of the image. Here we
use this method to calculate the empirical mean value of
each column of the image. Because the pixel values are
stored in matrix format by using matrix conversions.
This makes the value as the index values and it stores in
the database. The mean value is calculated from each
column value that is compared with the stored values in the
database.

This method is used for the reduction of gray scale

images into binary images and also it performs clustering
based image thresholding process. This algorithm assumes
that the image contain two classes of pixel values
(foreground pixel and background pixel). The optimum
threshold value is calculated by separating the two classes of

R.M.K College of Engineering and Technology, Chennai

Page3
 PAPER CONTEST - 2020 28th March 2020
the pixel.[5] So that the combined spread value is minimal
or equivalent as their inter class variance is maximal. This
method gives good performance , if the histogram values is
assumed to have bimodal distributions and it assumes to be
deep and sharp valley between the two peaks.

C. Reporting to the server

The next part of the project is reporting of potholes to the
server. The components consists of WIFI module EPS8266 ,
UART TTL logic and web server. If a pothole is detected,
the complaints based on the potholes are sent to the highway
department which is stored in the database. The complaints
are registered using IOT device. Each an every time the
complaints made by public, their location will be updated
automatically to the highway department.The message
contains the location of potholes and time. This will help for
the further road repairment.

III. PROTOTYPE AND TESTING

Here we considered two test cases, one is the ideal setup
and the other is the non-ideal setup. In the ideal setup we
consider, no other vehicles, obstacles and other pedestrain
on the road. In non-ideal setup it is normal road condition
where every obstacles are considered. The test was done to
detect TP (true positive), TN (true negative), FP (false
positive), and FN (false negative).
For the ideal set-up, two dummy potholes were
considered. The set-up used assumed a clear road without
any obstructions, no other vehicles, and no pedestrians as
seen on the sample images. The results of the experimental
trials done with four different speeds having thirty trials
each showed that system was successful in detecting
D. System Integration potholes. At the same time, the system was also successful
The main aim of the system is to detect the potholes and to displaying the detected potholes in the web server along
report the potholes to the high way department for the with its corresponding details such as the time and location.
welfare of the public safety. First the prototype is switched The improvement Vs old and new prototype is shown below.
on, the camera starts capturing the images of the road in
video format, for each second per frame. It checks for
potholes if pothole is detected, it process the pothole by
using detection algorithm. And segments the pothole image
with the location and dimensions and sends to the server.

R.M.K College of Engineering and Technology, Chennai

Page4
 PAPER CONTEST - 2020 28th March 2020

The segmentation step is to separate a pothole region

from the background region by transforming an original
color image into a binary image using the histogram of an
input image. HST (Histogram Shape-Based Thresholding),
maximum entropy, and Otsu [16] can be used for this
transformation into a binary image. In this study, an input
image is transformed into a binary image using HST [17].
In this table, in order to represent accurate detection
performance, the number of true positives (TP, correctly
detected as a pothole), false positives (FP, wrongly detected
as a pothole), true negatives (TN, correctly detected as a
nonpothole), and false negatives (FN, wrongly detected as a
nonpothole) was counted manually.

IV. CONCLUSIONS
The smart detection and reporting of pothole system was
able to operate successfully. The camera was efficiently
used for the detection of potholes where several algorithms
like erosion, dilation, Otsu thresholding, and contouring
were used for image processing. This, pothole detection
method based on 2D road images was proposed for
improving the existing method and designing a pothole
detection system to be applied to ITS service and road
management system. The deployment of automatic detection
system did not use any laptop, desktop nor Smartphones.
The system produced an average accuracy, sensitivity, and
The formula used to calculate the accuracy, sensitivity, specificity within the expected output and the success rate of
specificity, and success rate of the system are shown in sending reports of detection was found to have no errors.
Equations 1 to 4. The overall result of the system’s Regarding the experiment results, the proposed method
performance is shown in Table 3. For the ideal set-up, it reaches an overall accuracy of 93.75%. However, there are
shows a 100% results for all categories both for detection some limitations in the proposed method. Potholes may be
and reporting, see Figure 5 for the captured images which falsely detected according to the type of shadow and various
was displayed in the web server. shapes of potholes. Thus, in order to more accurately detect
However, for the non-ideal set-up false detections were potholes, it is necessary to use images from not only a single
present in the system, see Figure 6 for the captured sensor but also additional sensors and to add to the proposed
images. These errors were results of various factors during method more features for these sensors.
the testing. Factors that caused false detections were due to
the presence of other vehicles on the road, reflection of the
object was falsely detected as a pothole, objects such as ACKNOWLEDGMENT
water puddles, uneven colored pavements, and other road The heading of the Acknowledgment section and the
distresses also caused false detection. As for the success References section must not be numbered.
rate of sending reports of detected potholes, all detection
was successfully uploaded and displayed to the web server REFERENCES
for an average success rate of 100%. [1] Dong-Won Jang, Rae-Hong Park, "Pothole detection using spatio-
temporal saliency", IET Intelligent Transport System, August 2016.
[2] Alfandino Rasyid, Mochammad Rifki Ulil Albaab, Muhammad
Fajrul Falah, Yohanes Yohanie Fridelin Panduman and Alviansyah
Arman Yusuf, "Pothole Visual Detection using Machine Learning
Method integrated with Internet of Thing Video Streaming
Platform", IES International Electronics Symposium, 2019.

R.M.K College of Engineering and Technology, Chennai

Page5
 PAPER CONTEST - 2020 28th March 2020
[3] SungWon Lee, SeokJin Kim, Kwang Eun An, Seung-Ki Ryu and
Dongmahn Seo, "Image Processing-based Pothole Detecting System
for Driving Environment",IEEE International Conference on
Consumer Electronics (ICCE), 2018.
[4] Panop Khumsap, Nalina Phisanbut, Pirawat Watanapongse and
Punpiti Piamsa-nga,"Novel Feature Extractions for Reflection,
Alligator Cracks and Potholes Road Surface Classification", IEEE,
2018.
[5] Mae M. Garcillanosa, Jian Mikee L. Pacheco, Rowie E. Reyes,
Junelle Joy P. San Juan,"Smart Detection and Reporting of Potholes
via Image-Processing using Raspberry-Pi
Microcontroller",IEEE ,2018.
[6] M. Uzair Ul Haq, Moeez Ashfaque, Senthan Mathavan ,"Stereo
Based 3D Reconstruction of Potholes by a Hybrid, Dense Matching
Scheme", IEEE Sensors Journal, 2019.
[7] Van Khang Nguyen and Eric Renault, "Cooperative Sensing and
Analysis for a Smart Pothole Detection", IEEE, 2019.
[8] Tran Duc Chung, M.K.A. Ahamed Khan,"Watershed-based Real-
time Image Processing for Multi-Potholes Detection on Asphalt
Road",International Conference on System Engineering and
Technology(ICSET), October 2019.
[9] Kwang Eun An, Sung Won Lee, Seung-Ki Ryu and Dongmahn
Seo,"Detecting a pothole using Deep Convolutional Neural Network
models for an adaptive shock observing in a vehicle driving" ,IEEE
International Conference on Consumer Electronics (ICCE), 2018.
[10] Yifan Pan, Xianfeng Zhang, Guido Cervone, and Liping Yang,
"Detection of Asphalt Pavement Potholes and Cracks Based on the
Unmanned Aerial Vehicle Multispectral Imagery", IEEE Journal in
applied earth observations and remote sensing, 2018.
[11] Amita Dhiman and Reinhard Klette,"Pothole Detection Using
Computer Vision and Learning",IEEE Transactions on Intelligent
transportation systems, 2019.
[12] Shambhu Hegde, Harish V. Mekali, Golla Varaprasad,"Pothole
Detection and Inter Vehicular Communication", IEEE International
Conference on Vehicular Electronics and Safety (ICVES), 2014.
[13] Sudarshan S Rode, Shonil Vijay, Prakhar Goyal, Purushottam
Kulkarni, Kavi Arya ,"Pothole Detection and Warning System:
Infrastructure Support and System Design", International
Conference on Electronic Computer Technology,2009.
[14] Zhang Yiyang ,"The Design of Glass Crack Detection System Based
on Image Preprocessing Technology",IEEE ,2014.
[15] Na WEI , XiangMo ZHAO,Tao Wang ,"Beamlet Transform Based
Pavement Image Crack detection", International Conference on
Intelligent Computation Technology and Automation, 2010.
[16] N. Otsu, “A threshold selection method from gray-level his-
tograms,” IEEE Transactions on. Systems, Man, and Cybernetics,
vol. 9, no. 1, pp. 62–66, 1979.
[17] C. Koch and I. Brilakis, “Pothole detection in asphalt pavement
images,” Advanced Engineering Informatics, vol. 25, no. 3, pp.
507–515, 2011.

R.M.K College of Engineering and Technology, Chennai

Page6