Measurement Science and Technology

Meas. Sci. Technol. 34 (2023) 095017 (14pp) https://doi.org/10.1088/1361-6501/acdb8d

Road potholes detection from MLS point

clouds
Xinjiang Ma1, Dongjie Yue1,∗, Siyuan Li1, Dongjian Cai2 and Yi Zhang3
1
School of Earth Sciences and Engineering, Hohai University, Nanjing 211100, People’s Republic of
China
2
Jiangsu Provincial Key Laboratory of Environmental Science and Engineering, Suzhou University of
Science and Technology, Suzhou 215000, People’s Republic of China
3
The First Company of China Eighth Engineering Division Ltd, Ji’nan 250100, People’s Republic of
China

E-mail: [email protected]

Received 23 February 2023, revised 2 April 2023

Accepted for publication 5 June 2023
Published 14 June 2023

Abstract
The extraction of pavement damage information is one of the major difficulties in the
application research of mobile laser scanning point cloud data. To address the problem of
inaccurate detection results by using only relative distance to detect potholes, this paper
proposes a novel pothole detection method that combines directed distance and skewed
distribution. Firstly, the rapid positioning of the pothole is realized by the directed distance,
which is calculated from the points and the local fitted plane. And monomerization and
denoising of potential potholes are achieved by density clustering. Then, the new accurate plane
is fitted by the surrounding pavement points of the potential pothole to obtain accurate directed
distances. The negative skewed distribution of the directed distance histogram and the skewness
coefficient are used for the accurate determination of the pothole. Finally, the three-dimensional
geometric features of the pothole are extracted. Experiments were carried out on a road with
poor road conditions. The experimental results validated the effectiveness and practicality of the
proposed method. It can achieve automatic detection of potholes with different shapes and
deformation degrees, and has effectively improved the efficiency of automatic road inspection.
Keywords: MLS, point clouds, road pothole, plane fitting, directed distance

(Some figures may appear in colour only in the online journal)

1. Introduction maintenance phase. Efficient and automated pavement meas-

urement and inspection technology has become an important
As an important carrier of transportation, the road plays an need for the road traffic industry [3, 4]. After a long period of
important role in promoting economic development [1, 2]. erosion by the natural environment and crushing by vehicles,
By the end of 2021, the total road mileage of China reached the pavement gradually produces multiple types of damage
5280 700 km. Among them, the mileage of roads in need of such as cracks, ruts and potholes. Among pavement damages,
maintenance reached 5251 600 km, accounting for 99.4% of potholes are serious road structural damage, which seriously
the total road mileage. The development of China’s roads has affects the aesthetics of the road, the stability of the vehicle, the
shifted from a large-scale construction phase to a large-scale comfort and safety of the driver. Timely and accurate detec-
tion and maintenance of potholes can ensure the safety of road
travel while minimizing maintenance costs. This is of critical
importance in assessing the extent of pavement damage and
∗
Author to whom any correspondence should be addressed. developing road maintenance plans accordingly.

1361-6501/23/095017+14$33.00 Printed in the UK 1 © 2023 IOP Publishing Ltd

Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

The traditional manual inspection is carried out by the is inexpensive and easy to implement. Moreover, only one-
inspector driving the vehicle and taking photos to record dimensional signal processing is involved, so the processing
road damages through visual discrimination. Manual inspec- speed is fast. However, the biggest shortcoming of this type
tion has many shortcomings. For example, the safety of the of method is the existence of missed detections, resulting in
inspector cannot be fully guaranteed, a large amount of man- inaccurate information such as the maximum depth and area
power and cost is required, the inspection speed is slow and of potholes.
inefficient, and the accuracy of the inspector’s field records The image-based methods are currently the most widely
is low. With the rapid development of science and techno- used methods in automated pavement damage detection [9].
logy, researchers have shifted the research focus to intelli- This type of method uses features such as grayscale, contrast
gent inspection with higher safety. Currently, identification of and local independence to detect potholes by image processing
potholes from pavement images is one of the main focuses of techniques. The commonly used traditional image processing
the research. Although a great number of research achieve- techniques mainly include image enhancement [10–12], mor-
ments have been achieved, three-dimensional (3D) features phological processing [13, 14] and image segmentation
such as the depth and volume of potholes cannot be obtained [15–17]. With the development and progress of algorithms,
from two-dimensional images. This results in a less accurate machine learning methods represented by support vector
assessment of pavement condition. machines [18, 19] and deep learning methods such as Faster R-
The mobile laser scanning (MLS) technology, represen- CNN [20–22], YOLO [23–25] and U-Net [26–28] are widely
ted by vehicle-mounted light detection and ranging (LiDAR), used for pavement damage detection and segmentation extrac-
can acquire high-precision 3D point cloud data of roads in a tion. In a world, the technique of using images to detect
long-time sequence without affecting normal traffic, providing potholes has been relatively mature. However, potholes detec-
important 3D basic data for pavement condition detection and tion from images always suffers from interference factors such
assessment. However, road point clouds acquired by MLS are as light, shadow, debris, water and oil stains, resulting in inac-
disordered, heterogeneous, complex and massive, and there is curate results such as location, area and depth of potholes.
no special explicit information available for pothole identific- These make it difficult to accurately evaluate the condition of
ation. Therefore, it is still a challenging task to detect potholes the pavement.
and extract its relevant information from MLS point clouds. The 3D reconstruction-based methods use ground cameras
Based on the 3D shape and spatial distribution characterist- or unmanned aerial vehicles to acquire pavement images, and
ics of potholes and their neighboring pavement point clouds, reconstruct the continuous pavement images into 3D point
a method for automatic detection of potholes from MLS point clouds by using methods such as binocular stereo vision
clouds was proposed. The pothole is accurately detected by [29, 30] or structure from motion [31–34]. And then the
the directed distance and the skewed distribution features of potholes are detected based on the reconstructed 3D point
the point clouds in the local region. The 3D geometric fea- cloud. The essence of this type of method is to make up
tures of the extracted potholes provide detailed basic data for for the lack of elevation information in the images, and the
pavement condition monitoring and evaluation. detection method is similar to the method based on 3D scan-
The remainder of this paper is organized as follows. ning. Although such methods can obtain effective 3D inform-
Section 2 mainly reviews the related studies on road potholes ation, errors are prone to occur in the image matching pro-
detection. Section 3 proposes the road potholes detection cess, resulting in unstable detection results. Therefore, this
method. Section 4 describes experimental performance of our type of method is not suitable for large-scale pavement pothole
method and section 5 draws some conclusions. detection.
The method of potholes detection using 3D laser scanning
point clouds can be divided into point-based, line-based, and
2. Related studies surface-based according to the processing unit. ¬ Point-based
detection methods filter potholes mainly by the curvature, nor-
With the development of detection equipment and algorithms, mal, height difference and derived geometric features of point
the pothole detection is transforming from manual inspection clouds. This kind of method is relatively simple, and can detect
to intelligent detection. The research fields of intelligent detec- obvious and regular potholes under relatively good road con-
tion mainly focus on vibration anomaly detection, image pro- ditions. In the case of medium or poor road conditions, the
cessing, 3D reconstruction and 3D laser scanning point clouds. detection results are inaccurate and there are many erroneous
The vibration anomaly detection-based method use the results. ­ The line-based detection method, first obtain the
abnormal change of signal from rangefinder or accelera- scan lines or profile lines according to the idea of dimension-
tion sensor to detect the location of potholes [5–8]. For ality reduction. Then lines or curves are fitted, and the pothole
example, Madli et al [8] combined the global positioning points are detected based on height difference or anomaly
system receiver, the ultrasonic sensors and the global stand- detection methods [9, 35–37]. Although this kind of method
ards for mobile communication modem for potholes detection. can obtain the depth of the pothole, it is difficult to detect
Ultrasonic sensors are used to measure the distance between the edge points of the pothole and is sensitive to noise on a
the car body and the road surface. If the distance at a cer- single line. ® The surface-based detection methods first fit
tain place is greater than the threshold, the place may be a local planes or surfaces. Then the pothole region is filtered
pothole. In short, the equipment used in this type of method by the point-to-surface directed distance threshold [38–41].

2
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

This kind of method integrates the overall 3D features and segmentation in road scenes [34, 42–44]. Based on the flat-
local aggregation features of potholes, and the core lies in the ness and continuity of the pavement and the separation of
method and range of surface fitting, as well as the calcula- the curbs as the road boundaries, the MLS pavement point
tion of directed distances. And it is consistent with the idea clouds are segmented through verticality and hierarchical
of manual precision verification and is an important way to clustering.
automate the detection of potholes. First of all, the k-dimensional (KD) tree is constructed for
In summary, although 3D point clouds can completely the point cloud to quickly search the neighboring point clouds
reproduce the 3D apparent shape of potholes, it is still difficult of the point p. The principal components analysis is used
to automatically detect potholes from large-scale road point to obtain the eigenvectors of the point clouds in the spher-
clouds due to the complex and diverse pavement environment, ical neighborhood of p, and the eigenvector with the smallest
scattered distribution of potholes, few regions, and large shape eigenvalue is used as the normal vector N of p. Then the vertic-
differences. ality V of p is calculated using equation (1) [45, 46]. The point
clouds rendered by verticality are shown in figure 2. Through
threshold filtering, point clouds in flat regions with small ver-
3. Method ticality such as pavement, are retained, while point clouds
in regions with large verticality such as curbs and vehicle
Potholes are present in the pavement in the shape of bowls or
tires are deleted. Finally, many clusters are obtained through
in irregular shapes. Potholes have a distinct depressed feature,
hierarchical clustering based on Euclidean distance. And the
as shown by their height being lower than the height of the nor-
cluster with the largest number of points is the pavement point
mal pavement around them. Moreover, a pothole has obvious
clouds
relative independence. These spatial features are manifested in V = 1 − |NZ | (1)
the point clouds by the negative directed distances from points
in the potholes to the pavement, as well as the negative skewed where NZ is the component of the normal vector N on the Z
distribution of the directed distance histogram between the axis.
potholes and the surrounding pavement point clouds.
Based on the above features, the directed distances of the
point clouds and their statistical distribution features are used 3.2. Automatic potholes detection
to detect potholes. As shown in figure 1, the proposed method 3.2.1. Potential potholes positioning. The pavement point
consists of three main steps: pavement point clouds segmenta- clouds are projected vertically onto the XOY plane and divided
tion, automatic potholes detection and geometric features cal- into regular grids, with each grid considered as the smallest
culation. First, the pavement point clouds are segmented out unit of the fitted plane. Compared with LS, RANSAC is less
from the complex MLS point clouds by verticality and hier- disturbed by noise and is a commonly used robust estimation
archical clustering. Then, the pavement is divided into regu- method. However, there is a certain probability that RANSAC
lar grids. The method combining random sample consensus obtains accurate results. RANSAC is probabilistic, and the
(RANSAC) and least square (LS) is used to fit a local plane probability of getting an accurate result is not 100%. So, the
to the point cloud within each grid. The positioning and ini- method of combining RANSAC and LS (R-L method) is used
tial filtering of potential potholes are achieved by the dir- to fit the plane for the robustness of the results and to prevent
ected distance from the points to the local plane. And the the interference of noise. Suppose the formula for the plane
method of density-based spatial clustering of applications with model S is as follows,
noise (DBSCAN) is used for monomerization and denoising
of potential potholes. After that, using the surrounding pave- A·X+B·Y+C·Z+D = 0 (2)
ment point clouds of potential potholes as input data, the above
method is used to fit the new plane to obtain new accurate where A, B, C and D are the parameters of the plane model.
directed distances. And the potholes are accurately judged by The basic steps of the plane fitting are as follows, corres-
the negative skewed distribution of the directed distance his- ponding to figure 3.
togram and the skewness coefficient (SK). Finally, based on ¬ The smallest data set Q for plane model estimation is
the regional connectivity and independence of the pothole, the randomly selected from the data set without duplication.
3D spatial geometric features such as depth, projected area and ­ The data set Q is used to fit the plane S, and the paramet-
repair volume are calculated by means of height difference and ers A, B, C and D are obtained.
projection. ® The distances of all points to the plane model S are cal-
culated. The points within the allowed threshold are marked
as inliers and the ratio of inliers is calculated.
3.1. Pavement point clouds segmentation
¯ To compare the proportion of inliers in the current model
The MLS point clouds of road scenes contain not only road with the proportion of inliers in the previous best model,
point clouds such as pavement and road curbs, but also a and to record the inliers corresponding to the larger one and
large number of non-road point clouds such as vehicles, trees, corresponding model parameters.
light poles, buildings and noise. The complete and accurate ° To repeat ¬–¯ for N iterations or, until the end of the
pavement is a prerequisite for extracting potholes, and there proportion of inliers in the current best model is greater than
have been some research results on pavement point clouds the threshold value. And to get the inliers data.
3
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

Figure 1. Workflow of the proposed method.

Figure 2. The point clouds rendered by the verticality.

± To use the inliers data to fit the plane S by LS and then 3.2.2. Potential potholes monomerization. The potential
get the plane model parameters. pothole regions not only have clustered points, but also have
After getting the parameters of the local plane, the directed sporadically distributed points. As shown in the red region
distance from the point to the local plane is calculated. Figure 4 in figure 5(a). In addition, there are also cases where point
shows a section of the points and the fitted plane. The black clouds of the one pothole are divided into different grids. In
points are points on the section. The red line represents the fit- consequence, based on the local aggregation of the pothole,
ted plane and the blue lines represent the directed distances. all potential pothole regions are divided into different clusters
The calculation formula for the directed distance Ddir is as using DBSCAN. The scattered points are deleted to obtain the
follows agminated potential potholes. As shown in figure 5(b).
A · xi + B · yi + D
Ddir = zi + (3)
C 3.2.3. Determination of potholes. The points of potential
potholes are taken as seed points (interior points), and the nor-
where xi , yi and zi are the 3D coordinates of one point. mal pavement points around them are obtained by the region-
The down-concave feature of potholes determines that their growing of the spherical domain with the Euclidean distance
directed distances are negative. The point clouds with directed as a constraint. The normal pavement points are noted as the
distances less than the threshold ∆D are filtered as potential neighborhood points of the potential potholes (exterior points).
potholes. As shown in figure 6, the green points are interior points, and

4
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

Figure 3. Workflow of the plane fitting.

Figure 4. Diagram of the pothole profile, local plane and directed distance.

the red points are the exterior points, as well as the pavement where n is the amount of data, Di is the directed distance to
points. The black points are the pavement points. The blue a point, D̄ is the mean of the directed distances, and σ is the
circles represent the boundary of the spherical neighborhood, standard deviation of the directed distance.
and R is the radius of the spherical neighborhood. The local The directed distance histogram of the interior points and
plane is refitted using the exterior points and the R-L method. exterior points in the pothole region has a negative skewed
And then, the directed distances of interior points and exterior distribution and the SK is negative, as shown in figure 7.
points are recalculated to ensure the accuracy of the directed Therefore, the quantitative judgment of the pothole is realized
distances in the pothole regions. by the SK. If the SK is within the preset threshold, then the
The skewed distribution is a data distribution with an asym- region is a pothole, and vice versa, it is not a pothole.
metric left-right distribution curve, as opposed to a normal dis-
tribution. It can be divided into two types: positive skewness 3.3. Geometric features calculation
and negative skewness. The former curve has a long right-hand
side and a short left-hand side, while the latter curve has a long The points in the pothole with the directed distances less than
left-hand side and a short right-hand side. The SK is a special- the preset threshold ∆D are taken as the pothole points. In
ized measure of the degree of bias in the data distribution. It addition to calculating the depth and projected area of the
generally uses the ratio of the third-order central distance to pothole, the length, width and repair volume of the pothole
the third power of the standard deviation to measure the bias along the lane line are calculated to provide data reference for
of the data distribution, and is denoted by SK. It is calculated the amount of materials used in road rehabilitation.
by the following formula. ¬ The depth of a pothole is the maximum height difference
between the interior points of the pothole and its surround-
∑n 3
ing normal pavement points. Therefore, the maximum abso-
i =1 (Di − D̄) lute value of the negative directed distances from the interior
SK = (4)
n · σ3 points of the pothole is taken as the pothole’s depth ∆H.

5
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

Figure 5. Potential potholes monomerization.

Figure 7. Directed distance histogram.

Figure 6. Diagram of the interior points and the exterior points.

are also the boundary points of the pothole, and the red line
­ The projected area is the area of the pothole projected is the boundary of the pothole. Finally, the area enclosed by
on the horizontal plane. First, the pothole points are projec- its boundary is calculated, which is the projected area of the
ted onto the XOY plane. Then the method of Alpha-Shapes is pothole.
used to extract the boundary points from the pothole points ® The repair volume of the rectangular shape is calculated
and connect them into closed lines. As shown in figure 8, the according to the repair principle of ‘square patching for round
black and the red points are the pothole points, the red points potholes and slant potholes’. First, the local coordinate system

6
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

Figure 8. Boundary of the pothole.

Figure 9. Local coordinate system.

is constructed. As shown in figure 9, the driving direction of

V = (L + 0.2) · (W + 0.2) · ∆H. (6)
the lane line is the X axis, and the vertical direction of the lane
line is the Y axis. The dimensions of the pothole in the local
coordinate system are then calculated as its length L and width 4. Experiments
W. At the same time, to take the depth ∆H as the height of the
repair area. So, the area S of the bounding rectangle along the 4.1. Data
lane can be calculated according to the following equation
The road point clouds data are acquired using a MLS system.
S = L · W. (5) The length of the road is about 1.6 km, and the minimum point
spacing of the data is about 2 cm. The storage size of point
Since the repaired area needs to be 10–20 cm larger than the cloud data is about 2.2 G, with a total of 64 million points.
actual pothole size, the repaired volume V can be calculated As shown in figure 10, there are cement isolation piers in the
according to the following equation middle of the road. The scenes include pavement, road curbs,

7
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

Figure 10. Experimental data rendered by elevation and experimental results (black regions).

Table 1. Critical parameter values for the experiments. ¯ The radius of regional growth R, which represents the
Parameter Description Value range of normal pavement around the pothole. It can be one-
quarter to one-third of the width, calculated from the minimum
∆V Threshold of the verticality 0.1 circumscribed rectangle of the potential pothole. We recom-
Gside Side length of the grid 50 cm mend that 20 cm is a suitable value in the case of extensive
∆D Threshold of the directed distance −1 cm experimental analysis. That setting it to 20 cm leads to that the
R Radius of regional growth 20 cm pothole points and pavement points jointly show the skewed
SK Skewness coefficient (−3, 0)
distribution characteristics.
° The SK, which is used for accurate judgment of the
pothole. The value range of the SK for a negative skewness
green belts, sidewalk trees, transportation auxiliary facilities distribution is generally is (−3, 0). The SK of most potholes is
and vehicles. The road surface is the asphalt pavement, which around −1.5, which is obtained empirically.
is in poor condition due to years of use and the role of the nat- For other parameters or some existing algorithms embed-
ural environment. There are a lot of cracks and repairs, and ded in the proposed method, the specific values are also
some potholes and ruts. In order to better illustrate the detec- affected by the point density. In the stage of pavement point
tion results, the potholes automatically extracted using the pro- clouds segmentation, the spherical neighborhood radius and
posed method, the black areas in figure 10, are superimposed the Euclidean distance in hierarchical clustering can both have
on the original road point clouds. initialization values of 0.2 m, which is similar to the road
curb elevation. In the stage of potential potholes positioning,
4.2. Parameters
the parameters of RANSAC can use the default values of the
PCL library. In particular, it is acceptable that the distance
The critical parameters and values used in the experiments are threshold is set to 0.01 m. In the stage of potential potholes
listed in table 1. And most of them were specified empirically. monomerization, DBSCAN can be used to filter deep cracks
¬ The threshold of the verticality ∆V is used to segment the and smaller pothole in accordance with requirements. In the
pavement point cloud. That setting it to 0.1 can remove non- stage of geometric features calculation, the radius ɑ used in
pavement point clouds, as well as point clouds at the junction the Alpha-Shapes needs to be greater than the point spacing
of pavement and curbs or tires, when the point spacing is less and the default value is 0.05 m. It cannot be too large because
than 10 cm. that huge value of the radius results in the projection area of
­ The side length of one grid Gside determines the range the convex hull.
of local plane fitting. Compared to the large-scale pavement For different road conditions and pothole shapes, the cor-
region, potholes are small targets and exhibit depressions only responding values of parameters described above can be used
in a localized range. That setting it to 50 cm can effectively as initial values. Specially, the Gside is inversely proportional
weaken the influence of road cross slope and longitudinal to the road conditions. In good road condition, the Gside can
slope. The value that is too large or too small may cause the be increased appropriately, but not more than 1 m. As the
potholes to be ignored. road condition deteriorates, the Gside is not less than 0.5 m.
® The directed distance threshold ∆D, which is used to In addition, the ∆D directly affects the quantity and depth of
filter the pothole points. Potholes with different depths can be extracted potholes. The smaller the ∆D, the deeper the extrac-
extracted by adjusting the parameter. Setting it to −1 cm, and ted pothole. From the top view, the shapes of potholes appear
points whose directed distance smaller than this threshold are to be approximately circular or elliptical due to the vehicle
considered as pothole points. and natural environment, and the radius ɑ directedly affects

8
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

Figure 11. Experimental results.

Figure 12. Partial top view (rendered by directed distance).

the projected area. The larger the radius ɑ, the closer the pro- of the algorithm in the paper is about 18 min. Figure 11
jected area is to the area of the convex hull. shows the overall experimental results. The bottom image
is the experimental data rendered by true color, and the red
or yellow regions are the potholes automatically detected by
4.3. Results
the proposed method. We selected some results for present-
The experiments were performed on a 3.2 GHz Intel Core ation and analysis. Figures 12 and 13 are the partial top
i5-11320H processor with 16 GB RAM. The running time views and lateral views of the blue arrow pointing regions in

9
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

Figure 13. Partial lateral view (rendered by directed distance).

figure 11, respectively, rendered by the directed distance. The 4.4. Analysis
blue–green points are the interior points of the potholes and
In order to verify the effectiveness and accuracy of the pro-
the red–yellow points are the pavement points.
posed method, and for security seasons, we inspected the road
It can be seen from the experimental results that the pro-
in the field and recorded the location, depth, length and width
posed method has good performance in road pothole auto-
along the lane travel direction of some potholes. Furthermore,
matic detection. Whether it is a regularly pothole or an irreg- we made a comparison with the method proposed by Liu et al
ularly pothole, the proposed method can detect its position [37]. The extraction results of three methods, the absolute
and internal region. For example, the potholes in the region deviations and relative deviations between two methods’ res-
A, H and I, which are surrounded by the flat pavement. Their ults and manual results are shown in table 2. And figure 14
shapes are relatively regular, and the 3D overall shape are shows the comparison of deviations in different geometric
similar to the bowl. And potholes in the region D, E, F, and features.
G are irregular in shape, and the elevations of the pavement Assuming that the results of manual measurements are
points around them are also different. The interior points and taken as the true values, the following results can be obtained.
exterior points of the pothole can be clearly distinguished by The depth of the most potholes extracted from the proposed
the directed distance, which facilitates the calculation of the method is slightly smaller than the depth measured manually
geometric features of the pothole. On the other hand, there (such as figure 14(a)), which may be due to two reasons. ¬
are also a few errors in the experimental results due to deep There is no point at the deepest position of the pothole, which
and wide cracks (such as region C), or speed bumps (such as is influenced by the density of the point clouds. ­ The influ-
region B). ence of the point cloud accuracy and the local plane results

10
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

Table 2. Geometric features of the potholes.

Pothole
Geometric features A D E F G H I

Manual 6.86 3.41 4.36 5.25 5.57 4.58 3.48

Ours 6.42 2.96 3.55 5.61 4.65 4.90 2.95
Absolute deviation −0.44 −0.45 −0.81 0.36 −0.92 0.32 −0.53
Depth/cm Relative deviation/% −6.4 −13.2 −18.6 6.9 −16.5 7.0 −15.2
Liu’s [37] 6.38 2.86 3.63 5.58 4.55 5.02 3.69
Absolute deviation −0.48 −0.55 −0.73 0.33 −1.02 0.44 0.21
Relative deviation/% −7.0 −16.1 −16.7 6.3 −18.3 9.6 6.0

Ours 0.1668 0.0516 0.0445 0.1012 0.0486 0.0780 0.0236

Projected
Liu’s [37] 0.1764 0.0458 0.0412 0.0612 0.0411 0.0845 0.0295
area/m2
Absolute deviation 0.0096 −0.0058 −0.0033 −0.04 −0.0075 0.0065 0.0059

Manual 0.5350 0.3880 0.3587 0.6408 0.2789 0.3331 0.2165

Ours 0.5461 0.3534 0.3849 0.6648 0.2599 0.3575 0.2049
Length Absolute deviation 0.0111 −0.0346 0.0262 0.0240 −0.0190 0.0244 −0.0116
along the Relative deviation/% 2.1 −8.9 7.3 3.7 −6.8 7.3 −5.4
lane/m Liu’s [37] 0.5475 0.3265 0.3268 0.4186 0.2548 0.3454 0.2101
Absolute deviation 0.0125 −0.0615 −0.0319 −0.2222 −0.0241 0.0123 −0.0064
Relative deviation/% 2.3 −15.9 −8.9 −34.7 −8.6 3.7 −3.0

Manual 0.3886 0.2258 0.1940 0.1891 0.2247 0.3047 0.1534

Ours 0.4120 0.2427 0.1766 0.2131 0.2457 0.2902 0.1672
Width Absolute deviation 0.0234 0.0169 −0.0174 0.0240 0.0210 −0.0145 0.0138
along the Relative deviation/% 6.0 7.5 −9.0 12.7 9.3 −4.8 9.0
lane/m Liu’s [37] 0.4204 0.2127 0.1703 0.2089 0.2054 0.3158 0.1597
Absolute deviation 0.0318 −0.0131 −0.0237 0.0198 −0.0193 0.0111 0.0063
Relative deviation/% 8.2 −5.8 −12.2 10.5 −8.6 3.6 4.1

Manual 0.207 90 0.087 61 0.069 59 0.121 18 0.062 67 0.101 50 0.033 21

Area of the Ours 0.224 99 0.085 77 0.067 97 0.141 67 0.063 86 0.103 75 0.034 26
bounding Absolute deviation 0.017 09 −0.001 84 −0.001 61 0.020 49 0.001 19 0.002 25 0.001 05
rectangle Relative deviation/% 8.2 −2.1 −2.3 16.9 1.9 2.2 3.2
along the Liu’s [37] 0.230 17 0.069 45 0.055 65 0.087 45 0.052 34 0.109 08 0.033 55
lane/m2 Absolute deviation 0.022 27 −0.018 16 −0.013 93 −0.033 73 −0.010 33 0.007 58 0.000 34
Relative deviation/% 10.7 −20.7 −20.0 −27.8 −16.5 7.5 1.0

Manual 0.002 97 0.000 85 0.000 96 0.001 72 0.001 13 0.001 23 0.000 51

Ours 0.002 93 0.000 73 0.000 78 0.002 00 0.000 95 0.001 34 0.000 44
Absolute deviation −0.000 04 −0.000 13 −0.000 18 0.000 29 −0.000 18 0.000 11 −0.000 07
Repair
Relative deviation/% −1.2 −15.1 −18.5 16.7 −15.9 8.7 −14.4
volume/m3
Liu’s [37] 0.002 96 0.000 62 0.000 71 0.001 41 0.000 84 0.001 41 0.000 54
Absolute deviation −0.000 01 −0.000 23 −0.000 25 −0.000 31 −0.000 29 0.000 18 0.000 03
Relative deviation/% −0.3 −27.2 −26.2 −17.8 −25.9 14.6 6.3

in slightly smaller directed distances. Although there are devi- figures 14(c), (d) and (f)). Moreover, for relatively regular
ations from the results of manual, the two automated methods circular potholes, such as A, H and I, the areas of the bounding
are basically consistent in terms of depth and projected area rectangle extracted from the two automated methods basically
(such as figures 14(a) and (b)). These demonstrate the effect- remain the same (such as figure 14(e)). Based on the compre-
iveness of the proposed method. In addition, most potholes’ hensive analysis of multiple geometric features, it can be con-
absolute deviation of the length and width is about 3 cm, and cluded that the proposed method has results closer to manual
the relative deviation is within 10%. The reason may be the measurements and has stronger stability.
error of manual measurement or the influence of point cloud To sum up, it can be seen that the proposed method detected
density. Even so, similar to the repair volume, the proposed the potholes existing in the road and extracted the geometric
method has smaller deviations than other method (such as features, which verifies the real situation that the road is in

11
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

Figure 14. Comparison of deviations in different geometric features.

poor condition and proves the effectiveness and practicality of method can effectively detect road potholes, which provides a
the proposed method. new method for road damage information extraction and con-
tributes to the accurate evaluation of road damage conditions.
To extract other pavement damages such as ruts and sinkholes
5. Conclusions
from MLS point clouds is the direction for subsequent
research.
In respect to the issue of inaccurate directed distance in
the surface-based detection methods and incomplete extrac-
tion from MLS point clouds, this paper proposed an auto- Data availability statements
matic potholes detection method from MLS point clouds
based on the directed distance and the skewed detection. The data generated and/or analyzed during the current
The research purpose is to automatically detect potholes and study are not publicly available for legal/ethical reasons but
extract geometric features from the MLS point clouds. The are available from the corresponding author on reasonable
local down-concave feature was used for localization and pre- request.
liminary filtering of potential potholes by fitting local planes The data cannot be made publicly available upon publica-
and directed distance filtering, and then the local aggregation tion because they are owned by a third party and the terms of
feature was used for monomerization of potential potholes by use prevent public distribution. The data that support the find-
density clustering. The new local planes were refitted using ings of this study are available upon reasonable request from
the pavement points around the potential potholes to obtain the authors.
the accurate directed distances. And the negative skewed
distribution was used to decide whether it is a pothole. Finally, Acknowledgments
the geometric features of the potholes were automatically
extracted and compared with the manual field inspection The present study was supported by the Jiangsu Provincial
method. The experimental results show that the proposed Key Laboratory of Environmental Science and Engineering

12
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

(Grant No. Zd1801). The authors would like to thank Rufei [16] Wang P, Hu Y and Tian M 2018 Asphalt pavement pothole
Liu and Jiben Yang, the authors of [37] sincerely for provid- recognition and extraction based on image texture Appl.
ing experimental results and suggestions. The authors would Res. Comput. 35 1596–600
[17] Koch C and Brilakis I 2011 Pothole detection in asphalt
also like to thank the reviews for their valuable comments and pavement images Adv. Eng. Inform. 25 507–15
suggestions. [18] Lin J and Liu Y, 2010 Potholes detection based on SVM in the
pavement distress image 2010 Ninth Int. Symp. on
Distributed Computing and Applications to Business,
ORCID iD Engineering and Science (Hong Kong, China, 10–12
August 2010) (IEEE) pp 544–7
Xinjiang Ma  https://orcid.org/0000-0001-9195-4739 [19] Al A et al 2021 Pothole detection using machine learning
algorithms 2021 15th Int. Conf. on Signal Processing and
Communication Systems (ICSPCS) (13–15 December 2021)
References (IEEE) pp 1–5
[20] Wang W, Wu B, Yang S and Wang Z 2018 Road damage
[1] Marein B 2022 Colonial roads and regional inequality J. detection and classification with faster R-CNN 2018 IEEE
Urban Econ. 131 103492 Int. Conf. on Big Data (Seattle, USA, 10–13 December
[2] Tamba M, Krause J, Weitzel M, Ioan R, Duboz L, Grosso M 2018) (IEEE) pp 5220–3
and Vandyck T 2022 Economy-wide impacts of road [21] Yan B, Xu G, Luan J, Lin D and Deng L 2021 Pavement
transport electrification in the EU Technol. Forecast. Soc. distress detection based on faster R-CNN and
Change 182 121803 morphological operations China J. Highw. Transp.
[3] Arya D, Maeda H, Ghosh S, Toshniwal D, Mraz A, 34 181–93
Kashiyama T and Sekimoto Y 2021 Deep learning-based [22] Liu C, Li J, Gao J, Gao Z and Chen Z 2021 Combination of
road damage detection and classification for multiple pixel-wise and region-based deep learning for pavement
countries Autom. Constr. 132 103935 inspection and segmentation Int. J. Pavement Eng.
[4] Zhao L, Wu Y, Luo X and Yuan Y 2022 Automatic defect 23 3011–23
detection of pavement diseases Remote Sens. 14 4836 [23] Dharneeshkar J, Aniruthan S and Karthika R 2020 Deep
[5] De Z, Keppitiyagama C and Seneviratne G 2007 A public learning based detection of potholes in Indian roads using
transport system based sensor network for road surface YOLO The 5th Int. Conf. on Inventive Computation
condition monitoring Proc. 2007 Workshop on Networked Technologies (ICICT) (Coimbatore, India, 26–28 February
Systems for Developing Regions (Kypto, Japan, 27 August 2020) (IEEE) pp 381–5
2007) (ACM) pp 1–6 [24] Park S, Tran V and Lee D 2021 Application of various yolo
[6] Eriksson J, Girod L and Hull B 2008 The pothole patrol: using models for computer vision-based real-time pothole
a mobile sensor network for road surface monitoring The detection Appl. Sci. 11 11229
6th Int. Conf. on Mobile Systems, Applications, and Services [25] Hou Y, Zhang H, Gao Z et al 2022 Research and comparison
(Breckenridge, USA, 17–20 June 2008) (ACM) pp 29–39 of intelligent detection methods of pavement distress based
[7] Mednis A, Strazdins G and Zviedris R 2011 Real time pothole on deep data augmentation J. Beijing Univ. Technol.
detection using android smartphones with accelerometers 48 622–34
The 7th IEEE Int. Conf. on Distributed Computing in [26] Li D, Duan Z, Hu X and Zhang D 2021 Pixel-level recognition
Sensor Systems and Workshops (DCOSS) (Barcelona, of pavement distresses based on U-Net Adv. Mater. Sci.
Spain, 27–29 June 2011) (IEEE) pp 1–6 Eng. 2021 5586615
[8] Madli R, Hebbar S, Pattar P and Golla V 2015 Automatic [27] Guan J, Yang X, Ding L, Cheng X, Lee V C S and Jin C 2021
detection and notification of potholes and humps on roads Automated pixel-level pavement distress detection based on
to aid drivers IEEE Sens. J. 15 4313–8 stereo vision and deep learning Autom. Constr. 129 103788
[9] Pan N, Du Y and Yue J 2022 Automatic detection method of [28] Wei C, Li S, Wu K, Zhang Z and Wang Y 2021 Damage
pavement deformation distress based on point cloud data J. inspection for road markings based on images with
Tongji Univ. 50 399–408 hierarchical semantic segmentation strategy and dynamic
[10] Tang L, Zhao C and Wang H 2008 Fusion of multiple basic homography estimation Autom. Constr. 131 103876
PDE models for enhancing road surface images J. Image [29] Fan R, Ozgunalp U, Hosking B, Liu M and Pitas I 2019
Graph. 9 1661–6 Pothole detection based on disparity transformation and
[11] Shen Y, Dang J, Wang Y and Feng X 2014 A compressed road surface modeling IEEE Trans. Image Process.
sensing pavement distress image filtering algorithm based 29 897–908
on NSCT domain J. Optoelectron. Laser 25 1620–6 [30] Du Y, Zhou Z, Wu Q, Huang H, Xu M, Cao J and Hu G 2020
[12] Xu Z, Che Y and Min H 2018 Initial classification algorithm A pothole detection method based on 3D point cloud
for pavement distress images using features fusion Int. segmentation Twelfth Int. Conf. on Digital Image
Conf. on Intelligent Interactive Multimedia Systems and Processing (ICDIP 2020) (19-22 May 2020) (SPIE) vol
Services (Gold Coast, Australia, 20–22 May 2018) (Cham: 11519 pp 56–64
Springer) pp 418–27 [31] Tan Y and Li Y 2019 UAV photogrammetry-based 3D road
[13] Radopoulou S, Jog G and Brilakis I 2013 Patch distress distress detection ISPRS Int. J. Geo-Inf. 8 409
detection in asphalt pavement images Proc. 30th Int. Symp. [32] Ahmed A, Ashfaque M, Ulhaq M, Mathavan S, Kamal K and
on Automation and Robotics in Construction (Quebec, Rahman M 2021 Pothole 3D reconstruction with a novel
Canada, 11 August 2013) (IAARC) vol 30 p 1 imaging system and structure from motion techniques IEEE
[14] Wu L, Mokhtari S, Nazef A, Nam B and Yun H-B 2016 Trans. Intell. Transp. Syst. 23 4685–94
Improvement of crack-detection accuracy using a novel [33] Saad A and Tahar K 2019 Identification of rut and pothole by
crack de-fragmentation technique in image-based road using multirotor unmanned aerial vehicle (UAV)
assessment J. Comput. Civ. Eng. 30 04014118 Measurement 137 647–54
[15] Chu X, Wang R and Chu J 2003 Asphalt pavement surface [34] Bicici S and Zeybek M 2021 An approach for the automated
distress image recognition based on moment invariant extraction of road surface distress from a UAV-derived point
feature J. Jilin Univ. 1 1–7 cloud Autom. Constr. 122 103475

13
Meas. Sci. Technol. 34 (2023) 095017 X Ma et al

[35] Mizutani T, Yamaguchi T, Kudo T, Yamamoto K, Ishida T, [41] Chen X, Ma R and Liang H 2022 Extraction of pavement
Nagata Y, Kawamura H, Tokuno T, Suzuki K and potholes based on normal vector distance Comput. Syst.
Yamaguchi Y 2022 Quantitative evaluation of peeling and Appl. 31 222–9
delamination on infrastructure surfaces by laser signal and [42] Yang B, Dong Z, Liu Y, Liang F and Wang Y 2017
image processing of 3D point cloud data Autom. Constr. Computing multiple aggregation levels and contextual
133 104023 features for road facilities recognition using mobile laser
[36] Liao F, Ma R and Wang D 2022 Road pothole extraction scanning data ISPRS J. Photogramm. Remote Sens.
method based on RANSAC Comput. Syst. Appl. 126 180–94
31 230–7 [43] Yadav M, Lohani B and Singh A K 2018 Road surface
[37] Liu R, Yang J, Ren H, Bori C and Chenhao C 2022 Research detection from mobile LIDAR data ISPRS Ann.
on a pavement pothole extraction method based on Photogramm. 4 95–101
vehicle-borne continuous laser scanning point cloud Meas. [44] Gao J, Chen Y, Junior J M, Wang C and Li J 2020 Rapid
Sci. Technol. 33 115204 extraction of urban road guardrails from mobile LiDAR
[38] Sun Q, Cheng J and Tian H 2019 Retrieval of asphalt road point clouds IEEE Trans. Intell. Transp. Syst.
pavement distress using multi-scale features extracted from 23 1572–7
unmanned aerial vehicle LiDAR data J. Shihezi Univ. [45] Demantké J, Vallet B and Paparoditis N 2012 Streamed
37 1–11 vertical rectangle detection in terrestrial laser scans for
[39] De B, Di B and Fiani M 2020 Mobile laser scanning data for facade database production ISPRS Ann. Photogramm.
the evaluation of pavement surface distress Remote Sens. I-3 99–104
12 942 [46] Guinard S and Landrieu L 2017 Weakly supervised
[40] Ravi R, Habib A and Bullock D 2020 Pothole mapping and segmentation-aided classification of urban scenes from 3D
patching quantity estimates using LiDAR-based mobile LiDAR point clouds Int. Arch. Photogramm.
mapping systems Transp. Res. Rec. 2674 124–34 XLII-1/W1 151–7

14