Proceedings of the Third International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC 2019)

IEEE Xplore Part Number:CFP19OSV-ART; ISBN:978-1-7281-4365-1

Internet of Things Based Pothole Detection System

using Kinect Sensor
D. Ramesh Reddy G. Phani Chandra Goud Dr. C. D. Naidu
Assistant Professor,ECE M.Tech Student, VNR VJIET Professor, ECE
VNR VJIET, Hyderabad, India. VNR VJIET Hyderabad, India VNR VJIET, Hyderabad, India
[email protected] [email protected] [email protected]

Abstract— One of the approaches to diminish street mishap is difficulties of identifying, detailing and of potholes and
distinguishing the mounds and potholes present out and different impediments on our street systems.
about. The greater part of the mishaps prompting demise Potholes are bowl-molded openings out and about that
caused because of potholes, speed breakers, and alcoholic can be up to 10 creeps top to bottom and are brought about
driving. In this paper.The Internet of things based street by the mileage and enduring of the road. Roads with potholes
checking is proposed to recognize the potholes and
protuberances on streets through Kinect Sensor so we can
have turned out to be almost widespread in india. Truth be
maintain a strategic distance from the accidents. The told, it has turned out to be natural for drivers to swerve
proposition portrays one such street support utilizes Kinect through paths to dodge potholes, which makes close by
sensor, Raspberry Pi, with Internet Capabilities that are vehicles causing accidents.
associated with an Internet-of-Things. The proposed
arrangement brings ongoing information about the condition
As indicated by the information shared by states with the
of the street in a specific area which can be sent to nearby center, uttar pradesh recorded the most astounding number of
specialists to participate in further activities. pothole passing at 987. In maharashtra, the quantity of
passing’s (726) had almost 2multiplied. Haryana and gujarat
Keywords— Pothole, cloud; kinect sensor; Rasberry Pi; IoT. additionally admission in adequately. Haryana revealed 522
passings a year ago, while it had reported no such fatalities in
2016. Eight individuals passed on in delhi because of
I. INTRODUCTION
pothole-related mishaps in 2017. A similar city had seen zero
Problems due to the traffic are increasing drastically passings in 2016, which gives a reasonable picture of how
mainly due to the increase in population. Heavy population potholes are turning into a savage circumstance for some.
has lot of needs and works, which automatically leads to Andhra pradesh, kerala, odisha and west bengal are there
heavy usage of vehicles. Vehicles have become part of life. among the main 10 states in passings identified with potholes
The main problem in these highly developing cities is the in the nation.
population is growing but there is no proper maintenance of
the roads happening. Hence there is lot of road accidents A. Problem statement
taking place. These congestion roads lead to the cracks As we know that due to heavy rains, roads are getting
initially and in the next stage they form potholes which lead damaged. We can see that there are many potholes on the roads
to the damage of the vehicle when the vehicle suddenly which are very harmful that prevent safe, secure and reliable
gets into it. In few cases when the driver watches these transportation. Potholes are a never-changing problem. These
potholes just before getting into it and suddenly applies potholes may cause small damage to road users, and in a few
breaks to avoid damage to his vehicles suspension. That cases, it may even cause a life threat to human beings. A
driver may save his vehicles suspension but in that situation pothole initially begins as an imperceptible minuscule disorder
unknowingly he is confusing the driver who is coming on the road surface and it may also be due to the low-quality
exactly behind him. That vehicle from backside is going to
materials used while laying them. The local drainage system,
hit him. In few situations this chain reaction continues to 3
Heavy rainfalls, and high temperature may also be the reasons
to 4 vehicles making a lot of damage to the body parts of
the cars. in the backend for the pothole formation.
IoT allows sensors with some networking capacities to II. LITERATURE REVIEW
be embedded in everyday objects which enables them to
communicate and share the information with other objects Development of infrastructure that has the good challenge
in the physical and virtual worlds (Al-fuqaha et al., 2015; and creates the way to enter in to the outside world. A
zanella et al., 2014). Potholes and speed ramps which are network is employed for communication, trade, and social
the most common obstacles seen on roads. These road quality of a rustic. Road offers more safety once it’s
obstacles and disruptions in the surface condition of roads maintained properly additionally the surface separation also
are among the causative factors of street mishaps, gores cause an increasing range of accidents. The unhealthy road
and fatalities. This paper analyzes the prospects and condition is that the main cause for all the truck accidents and
lots of additional consistent with the analysis in [1].In

978-1-7281-4365-1/19/$31.00 ©2019 IEEE 232

Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on June 14,2020 at 09:38:13 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Third International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC 2019)
IEEE Xplore Part Number:CFP19OSV-ART; ISBN:978-1-7281-4365-1

weekends the road congestion increases a lot and the traffic surface in the physical environment.
maintenance price and also the negative effects on There are some challenges of Internet of Things such as
mechanical process of different vehicles on the roads has connectivity issues, power issues, secure issues, context
been detected. Different factors that have an effect on the processing during the process, cloud connectivity using
performance of the roads are 1)Due to the potholes on the internet.
roads which makes a snarl that become difficult and the
situations like traffic does not moves or it slowly moves 2)
poor substance and inappropriate moisture condition at
construction part. Road transportation is a very important
system which becomes sleek and also cozy movement for all
the passengers. The main disadvantage for the uneven of the
roads such as pits, potholes, and edge depression on a road
surface are LED which damages the vehicles on the roads
[2].
To enrich upon this Li et al. [3] introduced a fair price
effective solution which used for infrared optical laser line
projectors, a digital cameras and a multiple views on two-
dimensional theme to calibrating the lasers to plot more
aspect points within the digital cameras. Homji [4] in his
point of view, intelligent hollow repair vehicle uses machine
driven arrangement which consists of a wheel attached to
spring. When wheel passes, the irrelevancy in the spring is
calculated by an optical shaft encoder. Depending on the
calculated value, the pitch of the pothole on the particular
road will forecast. But the mechanical approach used in this Fig. 2. Block diagram of Proposed System
system is hardly useful on high speed vehicles.
The readings and graphs gained by using the accelerometer The car engine is ignited on start up. If the car is in motion,
is made to process with an algorithm to obtain the exact type the raspberry pi gets on and the kinect sensor is activated.
of problem. Latitude and longitude values of the position The kinect sensor detects the potholes based on the counters
exactly where these potholes are being detected are exactly and the depth of the grayscale reading on the roads and it
recorded using the GPS system which is built in androids captures the image and sends that detected pothole to the
system.[5].Rajeshwari madli et al. had undergone a serious rasberrypi and the same data is sent to the cloud (dropbox).In
survey and finally designed a very cost efficient pothole our system a location.txd file is parallel sent at the same time
system.[6] to discover the cracks and potholes present on the with the image. We have created a desktop file. In this we
roads, and in parallel process to exactly measure the pothole have a script file. This script file starts the application by
size if detected by the ultrasonic sensor which is attached to using the start method. Just by double clicking that desktop
it.. These detected potholes and the depth of these individual file we will be getting two windows. One will show the
potholes is made to upload in the cloud for further use. When kinect window i.e the kinect sensor output and secondly it
these potholes are detected then there is an alert beep sound shows the counter specific output in counter specific window.
in the form of flash messages to alert the driver in that path. In the cloud level i.e in our dropbox we have a folder
In the same way to get the accurate values the ultrasonic called “Potholes”. When we open that folder we will be
sensor must be made to be tested by changing the position of having the images with the text file below. Those both will be
the pothole under the car. at the same time stand. The car fitted with the kinect sensor
moves along a road network. The car is setup to identify
III. IMPLEMENTATION potholes and road obstacles.
This paper used an open hardware device and a prototype If the car is fitted with camera, it is configured to take the
to build an IoT enabled device to detect and manage image of pothole location and sends the pothole image with
potholes. The paper uses a Raspberry pi board to build an latitude and longitude values to the server. The car also
interactive and context aware system which can sense the senses pothole detection information locally and alerts the
driver to slow down.
3.1 Kinect Sensor
The kinect sensor is used to collect the data and it is a
wireless controller for gaming(XBOX). Kinect sensor which
contains a camera and an infrared camera sensor which is
used to detect the position and also the distance information
Fig. 1. Potholes from one another. Kinect sensor is more economical than any
other sensors and it helps to reduce the road examination cost

978-1-7281-4365-1/19/$31.00 ©2019 IEEE 233

Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on June 14,2020 at 09:38:13 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Third International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC 2019)
IEEE Xplore Part Number:CFP19OSV-ART; ISBN:978-1-7281-4365-1

where a separate individual has to send in this place. Where sent to the cloud and this can be used for further necessary
kinect sensor also have a feature that obtains the distance actions.
information by Time of Flight(TOF) method, in which the
distance is measured based on the time taken from the IR A. Flow Chart
light and back to the sensor.
Kinect sensor is mainly created to use in the Xbox
gaming and also in many other games. We are now trying this
kinect camera in our project so that it can give us a very high
quality image. Although the sensor is bit expensive than
the normal we bca m . This kinect sensor converts infrared
light to the pulse wave therefore this sensor can be used in
open spaces where other sensors can be affected by sunlight.
Kinect sensing element is connected 1 meter from the bottom
and it works at 30 Hertz frequency with a specified size of
640 X 480 pixels. Resolve depth image that has taken the
dimensions with the resolution of 1280X1024 pixels may be
taken through kinect sensing element at the speed of 10 hertz.
Wherever the sector of read is 58 degrees horizontal,45
degrees vertical and 70 degrees diagonal. Image acquisition
was done victinisation.The open kinect software system with
ubuntu linux surroundings. An extensive method is written so
as to process and act the science and characteristic options of
chuckhole. After noise filtering we have a tendency to get
chuckhole at intervals the image. Next, the sting of the
segmental objection the binary image is extracted.
The data that is obtained from the kinect sensing element
are often utilized by alternative users because it compiles
the info and sent to the authorities to require further way
action regarding the pothole prone region. Finally the
detected pothole at the intervals of the image on the roads is
obtained.

Fig. 4. Algorithm for pothole detection system

The pothole sent in the cloud can be used for the depth analysis
and estimate the amount of severity of the road condition.

Fig. 3. Depth images of Potholes identified at different locations using

IoT pothole detection system using Kinect Sensor

IV. ALGORITHM AND METHODOLOGY

The pothole detection system is powered by a battery present
in the car. The system continuously monitors the condition of
the road by using a kinect sensor. The pothole detection Fig. 5. Prototype of IoT Based pothole detection
system uses the kinect sensor to record the road condition and
In our laboratory, the pothole detection system prototype is set
to detect the potholes present on the road. The depth images
up with the cardboard and created dummy potholes to initially
are logged. If a pothole on the road surface is detected the
location of the pothole image and location of the pothole is

978-1-7281-4365-1/19/$31.00 ©2019 IEEE 234

Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on June 14,2020 at 09:38:13 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Third International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC 2019)
IEEE Xplore Part Number:CFP19OSV-ART; ISBN:978-1-7281-4365-1

work on the depth images of the kinect sensor. fig. 5 shows the
prototype setup for a pothole detection system.
V. EXPERIMENTAL RESULTS

Fig. 8. Result plotting of Different methods of pothole Detection

Different systems that are used to detect the potholes are shown
Fig. 6. IoT Integration with Pothole Detection Device in the fig. 8 accuracy of the system using kinect sensor is great
with compared to other systems using 2d LiDAR, camera,
The kinect sensor initially placed on a small robot for taking stereo vision and android systems.
pothole images of a road, the robot has a metallic chassis and
four wheels. the raspberry pi board and the power supply to TABLE 1. METHODS OF IMPLEMENTATION
kinect are through 24 volts lead acid battery present on the
chassis of the vehicle. a fig. 6 is an image of the pothole
detection prototype.
Authors Different Methods used to detect Potholes
Methods Equipment
Kinect Sensor
Moazzam et Using Kinect Evaluation test in Pakisthan And
al. Sensor Rawalpindi
Jouber et.al Using Kinect Kinect and USB Camera
sensor
Test Bench with Two Cameras
Staniek et.al Using Stereo Feasibility Study
Kinect Sensor Vision

Wang et.al Using Stereo Two Cameras

Vision Feasibility Study
Using 3D laser 3D Laser and Digital Camera
Li et al. Scanning
3D Laser (MENSi GS10)
Chang et.al Using 3D Laser Simulation data Set and Real Data
Scanning Set
Digital Highway Data Vehicle
Hou et.al Using Stereo (Two Computers GPS, DML,
Camera Line scan Camera)

Combined finding of the different research papers and

methods used to identify the potholes are shown in table1.
Fig. 7. Proposed System Integrated to Mobile Vehicle

IoT pothole detection system is integrated into the four-

wheelers to detect the potholes on the road is shown in fig. 7
kinect sensor is placed on the front side of the car to capture the
condition of the road. kinect sensor detects the potholes present
on the road and sends the data to the cloud along with the
location. IoT pothole detection system is powered by a car
battery. When the ignition is on the IoT module starts
monitoring the condition of the road as well as a location for
detection of potholes.
Fig. 9. Pothole Data in Drop Box Cloud

978-1-7281-4365-1/19/$31.00 ©2019 IEEE 235

Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on June 14,2020 at 09:38:13 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Third International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC 2019)
IEEE Xplore Part Number:CFP19OSV-ART; ISBN:978-1-7281-4365-1

Communication technology has increasingly contributed [12] Kumar, R. Praveen, and S. Smys. "A novel report on architecture,
to the quality of life and work of many individuals. Which protocols and applications in Internet of Things (IoT)." In 2018 2nd
provided the convenience, safe, comfort and several other International Conference on Inventive Systems and Control (ICISC), pp.
benefits to the people? sensors and communication
technologies were seen as driving force behind IoT enabled 1156-1161. IEEE, 2018.
devices which have the capability of sensing and processing
their environmental conditions. We are presently taking the
potholes as the image; because we don’t have any predefines
pothole size. Whenever the system encounters some potholes,
the system is in loop mode; the system immediately captures
that snapshot and updates it in the drop box. We can conclude
by saying that this system may help the government
organizations under the road assistance wing to maintain the
roads with good standards. This system also helps in clearing
the potholes as soon as possible because of the location
access.
VI. REFERENCES

[1] Jose M. Álvarez, Antonio M. López, Theo Gevers, Felipe

Lumbreras,“Combining Priors, Appearance, and Context for Road
Detection”,IEEETransactions on Intelligent Transportation Systems, vol.
15(3), pp.1168- 1178,2014
[2] Ki Ryu, Taehyeong Kim,and Young-Ro Kims,”Image-Based Pothole
Detection System fot ITS Service and Management System”,Research
Article,2015
[3] Meiqing Wu, Siew-Kei Lam, ThambipillaiSrikanthan, “Nonparametric
Technique Based High-Speed Road Surface Detection” IEEE Transactions on
Intelligent Transportation Systems, vol.16(2), pp. 874 –884, 2015.
[4] Li, Q., Yao, M., Yao, X and Xu, B. (2009): A real-time 3D Scanning
System for pavementndistortion inspection, measurement science and
technology,Pages 15702-15709.
[5] R. Adi, M. Homji, “Intelligent Pothole RepairVehicle”, Texas A & M
University, August 2005.[6] “Gps Status| Android Developers” [Online].
Available:http://developer.
android.com/reference/android/location/GpsStatus.html
[7] R. Madli, S. Hebbar, P. Pattar, and V. Golla, “Automatic Detection
andNotification of Potholes and Humps on Roads to Aid Drivers,” IEEE
Sensors Journal, vol. 15, no. 8, pp. 4313–4318, 2015.
[8] Yau Weng Kuan, Ng Oon Ee, Lee Sze Wei “Comparative Study of Intel
R200, Kinect v2, and Primesense RGB-D Sensors Performance Outdoors”
IEEE Sensors Journal, Vol. 19 , Issue: 19 ,pp. 8741 – 8750 , 2019.
[9] Wen-Shyong Yu ; Kuo-Yao Huang “Implementation of media player
simulator using Kinect sensors” 2017 International Conference on System
Science and Engineering (ICSSE), 2017.
[10] Byeong-ho Kang, Su-il Choi, “Pothole detection system using 2D LiDAR
and camera” 2017 Ninth International Conference on Ubiquitous and Future
Networks (ICUFN), 2017.
[11]Suma, Towards Sustainable Industrialization Using Big Data And
Internet of Tnings. Journal of ISMAC, 1(01), 24-37,2019.

978-1-7281-4365-1/19/$31.00 ©2019 IEEE 236

Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on June 14,2020 at 09:38:13 UTC from IEEE Xplore. Restrictions apply.