Identification of Road Defects with a Smart Camera

V. S. Seleznev, E. O. Antonova, A. V. Iluhin, R. A. Gematudinov, L. Yu. Isaeva

2022 Systems of Signals Generating and Processing in the Field of on Board Communications | 978-1-6654-0635-2/22/$31.00 ©2022 IEEE | DOI: 10.1109/IEEECONF53456.2022.9744393

Moscow Automobile and Road Construction State Technical University (MADI)

Moscow, Russia
[email protected]

Abstract— This article discusses the process of identifying equipped with modern software and hardware systems,
roadway defects used in automated mobile laboratories by including photo and video recording cameras [5].
searching for the edges of an object. The introduction of
automated systems using machine vision will significantly reduce
the laboriousness of the process of detecting defects in roads, will
make it possible to control at a higher level and increase the
speed of processing the results. During the development process
on LabVIEW software, the main algorithms for the interaction of
a smart camera with an on-board system were described. The
process of image binarization, its frequency filtering through the
Fourier transform is described. To get around this limitation, the
article discusses fast Fourier transform algorithms.

Keywords— System of identification, intellectual transport

systems, LabVIEW, identifying roadway defect, image processing
methods, machine vision, automated mobile laboratories

I. INTRODUCTION
Fig. 1. Scheme for measuring the dimensions of the pothole
In order to effectively control the transport and operational
properties of road surfaces, it is necessary to receive timely
information about road surface defects that form during their
operation [1, 2]. According to the defect classifier developed
by A.V. Rudensky all types of defects can be divided into
several groups.
The first group, this is a violation of the continuity of the
coating, these include breaks, cracks along the run lanes,
potholes, destruction of the edges, peeling and chipping of the
road.
The second group is a violation of geometric parameters,
these include subsidence, abysses, rutting, roadway waves and
shifts. A defect of the "pothole" type is defined as follows
(figure 1). To begin with, measurements (figure 2) of the Fig. 2. Measurement of potholes at the MADI Polygon
maximum size of damage are carried out in two directions
relative to the movement of the car: perpendicular and parallel.
After that, a ruler is installed in such a way that it completely
covers the defect in the roadway. The last step is to measure
the clearance.
"Cracks" are measured (figure 3) with a tape measure in
directions parallel and perpendicular to the axis of the
highway

II. THE MOBILE LABORATORY

To optimize the process of detecting defects and prompt
repair of the roadway, mobile road laboratories are used,
which are equipped with modern software and hardware
systems, including photo and video recording cameras. [4] To Fig. 3. Measurement of cracks at the MADI Polygon
optimize the process of detecting defects and prompt repair of
the roadway, mobile road laboratories are used, which are

978-1-6654-0635-2/22/$31.00 ©2022 IEEE

Authorized licensed use limited to: Odisha University of Technology and Research. Downloaded on August 29,2024 at 15:24:03 UTC from IEEE Xplore. Restrictions apply.
 The mobile laboratory "ADS-MADI" (Figure 4) was
developed by the Moscow Automobile and Road Construction
State Technical University (MADI) to collect information on
the technical and operational condition of Moscow roads.

Fig. 5. Initial image of a defect (crack)

Fig. 4. Mobile laboratory "ADS-MADI"

The laboratory is used to diagnose the transport and

operational condition of the pavements of the road network.
The machine fixes the longitudinal evenness of the coating
with the determination of the microprofile and the possibility
of recalculation into a normalized indicator in accordance with
the standards.
These laboratories have the advantage that all of the above
tasks are performed while the vehicle is moving and there is
no expert group. But at the same time, the disadvantage of
automated systems is that the availability of road transport in
the study area significantly affects the receipt of adequate data.
Fig. 6. Image binarization by the lower threshold of values
Carries out photo-video fixation of defects in the form of:
cracks, potholes, rutting, etc. The equipment includes
computer scanning modules using on-board and front cameras.

III. MACHINE VISION

In machine vision, any image pixel is characterized by two
geometric parameters and one brightness value. In accordance
with this, it is customary to distinguish between the geometric
and brightness components of the image, which in turn are
absolutely independent of each other. Based on this principle,
the idea of a separate analysis of these quantities arose, which
gave rise to the method of image binarization.
When processing an image, they often resort to the
threshold binarization procedure, which consists in dividing
pixels into black and white, depending on the selected Fig. 7. Image binarization by the upper threshold of values
threshold value. The result of image conversion is shown in
Figures 5-7. Adaptive binarization is based on the principles of
selecting a global or local histogram of the processed image,
As can be seen from Figures 5-7, threshold binarization which is selected based on the total brightness of the pixel
does not allow assessing the presence of a defect, since the matrix, which makes this method the most common and
image contains a large amount of noise that distorts its image. functional [7]. The search for the optimal segmentation
Since each defect will have its own HSB value, it is necessary threshold is based on the selection of the closest histogram to
to regularly select threshold values, which is labor -intensive the image curve. This threshold is calculated probabilistically
and not rational without the involvement of machine learning. based on the weight sum of the intensity of the normal
For more accurate defect detection, it is required to use a more distribution.
advanced binarization method, such as adaptive binarization
[9]. To separate the object of detection (defect) from the
background or noise (roadway), the most significant method
[6].

Authorized licensed use limited to: Odisha University of Technology and Research. Downloaded on August 29,2024 at 15:24:03 UTC from IEEE Xplore. Restrictions apply.
 The essence of the method lies in the fact that the threshold Thus, to implement the method, the Father needs to select
of the object and the background is calculated separately, due the intensity probability histogram (B), perform the initial
to which the root-mean-square error of segmentation of adjustment of the background and object pixel distribution
background pixels from object pixels is minimized. It is density, and select the optimal intensity threshold for the
customary to take the pixel intensity value as random, and its roadway Figures 8.
histogram as an estimate of the probability distribution
density. With known probability distribution densities, it is
possible to determine the image segmentation threshold as
accurately as possible [8].
The algorithm of this method can be described as the sum
of two dispersions δ 1 and δ 2 and expressed through equation
(1).

δ ω2 (t) = ω1 (t) O δ12 (t) + ω2 (t) O δ 22 (t) (1)

where σ is the standard deviation of the original image, δ1,

δ2 are the standard deviation of the object and background,
respectively. ω1, ω2, pixel distribution density of the object
and background, respectively. The pixel distribution density
and standard deviation are determined depending on the Fig. 8. The result of overlaying the processed image and the real
threshold value t. The probability distribution density is
The disadvantage of this method is that it is ideal for a
calculated based on the curvature of bit histograms (2), (3).
static array of pixels, however, mobile laboratories work with
l −1 streaming images, and therefore it is necessary to sample the
ω1 (t) =  p(i ) (2) video into individual images. The question of discretization
i =0 will depend on the speed of shooting detectors and the speed
of movement of the laboratory. To level these limitations, one
L −1
should apply the Fourier transform and the wavelet transform.
ω2 (t) =  p(i ) (3)
i =t Since there are a number of limitations with binarization,
for streaming image processing, it is more expedient to carry
Under two-rank minimization, the intra-class variance is out linear filtering transformations. One of the filtering
equivalent to the maximum inter -rank variance (4) methods is the Fourier transform, which displays any data set
δ b2 (t) = δ 2 − δ ω2 = ω1 ( µ1 − µT ) 2 + ω2 ( µ 2 − µT ) 2 = ω1 (t)ω2 (t)[ µ1 (t) − µ2 (t)]2 (4) in the form of simple tribometric functions such as cosine and
sine. This transformation allows you to determine the
From expression 4, we get that the arithmetic mean of the discreteness of streaming video without decomposing it into
rank μT is the sum of the array of pixels of the object μ1 and separate images and to give a criterion-based assessment of
the background μ2. Thus, the background (5) can be their weight in the total array.
distinguished from the object (6) and the sum of these values
The Fourier transform in mathematics is subdivided into
must satisfy the conditions of equation (7).
Fourier series, Fourier integration and discrete Fourier
l −1 transform, but in working with images the discrete Fourier
 i • p(i)
i =0
transform is used.
µ1 (t) = (5) The Discrete Fourier Transform transforms the brightness
ω1 (t) sequence of streaming image pixels into a finite sequence of
Fourier coefficients. The coefficients can be expressed in
L −1
terms of various variants of finite sums (10).
 i • p(i)
i =0 N N
µ2 (t) = (6)
ω 2(t) 2
2π ni 2
2π ni
xi = a0 +  an cos( ) +  bn sin( ) (10)
L −1 n =1 N n =1 N
µT =  i • p(i ) (7) It is also worth noting that with a discrete Fourier
i=0 transform, streaming video is represented as a discrete
Since the image is a static array of pixels, the correctness convolution, that is, as a set of pixel values located in a
of the calculations is confirmed through the following uniform matrix.
equalities (8) (9) The discrete Fourier transform applied to streaming video
performs the function of frequency filtering, namely, it helps
ω1µ1 + ω2 µ2 = µT (8)
to cut noise in the form of a sum of pixels [3].
ω1 + ω2 = 1 (9)

Authorized licensed use limited to: Odisha University of Technology and Research. Downloaded on August 29,2024 at 15:24:03 UTC from IEEE Xplore. Restrictions apply.
 When working with roadway defects, the discrete Fourier Fig. 11. Image processing program
transform allows you to remove noise leaving only the
damaged area, but since this analysis uses only the frequency A program has been developed using machine vision
part of the transformation, it should be noted that it reduces algorithms, which allows not only to increase the speed of
the spatial component of the defect, which increases the image processing, but also to distribute computing power in a
probability of detection error when analyzing small cracks. way that allows implementing the modern technical level.
Thus, the quality of image processing is significantly
improved, thereby reducing the probability of an error in
IV. RESULT detecting a defect in the roadway [10]. At this stage of
Based on the tasks set, a search was made for the optimal development, the geometric parameters of the defect, the
solution in the field of machine vision for the automated coordinates of the location of the defect on the road and other
search for pavement defects. By studying advanced computing technical parameters, as well as more complex defects for
technologies, National Instrument (figure 9-11) technical detection using the visual method of defects, were not taken
solutions were selected, which are optimal, due to the fact that into account.
they use SoC technology in their products, and the integrated
development environment allows you to use the potential of The developed system will increase the speed of
computing power at the required level. monitoring the roadway by reducing manual operations, and
as a result, it will reduce the labor intensity and cost of
maintaining the roadway in the normalized transport and
operational characteristics.

References
[1] A. A. Petyakin. Vnedrenie avtomaticheskoj sistemy vesogabaritnogo
kontrolya avtotransporta na dorozhnoj seti RF: perspektivy razvitiya.
Interv'yu s V. Rechickim. Dorozhniki. 2017. No. 3(11), pp. . 36-41.
[2] A. A. Adyev, D. I. Kazarinov, R. N. Minnihanov et al. Integrirovannaya
sistema kontrolya vesogabaritnyh parametrov, narushenij PDD i oplaty
za proezd. Sovremennye problemy bezopasnosti zhiznedeyatel'nosti:
intellektual'nye transportnye sistemy: Materialy IV mezhdunarodnoj
nauchno-prakticheskoj konferencii, Kazan', 25-26 fevralya 2016 goda.
Fig. 9. Appearance of the program on LabVIEW Kazan': Nauchnyj centr bezopasnosti zhiznedeyatel'nosti, 2016, pp. 11-19.
[3] A. V. CHkan. Povyshenie real'noj proizvoditel'nosti RVS pri reshenii
zadach cifrovoj obrabotki izobrazhenij s ispol'zovaniem bystrogo
preobrazovaniya Fur'e. Izvestiya YUFU. Tekhnicheskie nauki. 2020. No.
7(217), pp. 152-163. DOI 10.18522/2311-3103-2020-7-152-163.
[4] D. Amanatidis, M. Dossis & I. Androulidakis. Image contour
segmentation in hardware. Radio Electronics, Computer Science,
Control, no. 4. 2015.
[5] D. A. Priluckij, A. P. Gusev, G. G. Znajko. Apparatnyj modul'
potokovoj obrabotki vysokochastotnogo videoizobrazheniya. Voprosy
radioelektroniki. 2018. No. 2, pp. 112-119.
[6] P. Yu. Kostenko, V. V. Slobodyanyuk, I. L. Kostenko, «Metod
podavleniya shuma na izobrazhenii v obobshchennom fazovom
prostranstve s uluchshennym pokazatelem prostranstvennoj
razreshayushchej sposobnosti», Izvestiya vuzov. Radioelektronika, vol.
62, no.. 7, ppp. 443-452, 2019.
[7] A. Ilyukhin, V. Seleznev, E. Antonova, M. Abdulkhanova, E. Marsova.
Fig. 10. Image processing result Implementation of FPGA-based camera video adapter for vehicle
identification tasks based on EUR 13 classification. IOP Conference
Series: Materials Science and Engineering. P. 6.
[8] A. V. Ilyukhin, I. I. Zaripova, E. V. Marsova, V. S. Seleznev. Digital
automation devices. Moscow: Moscow Automobile and Road
Construction State Technical University (MADI).
[9] V. S. Seleznev, E. O. Antonova. Identification of advantageous means of
a camera with a combined FPGA Science and technology in the road
industry: Conference Proceedings. Moscow: Moscow Automobile and
Road Construction State Technical University (MADI), 2021, pp. 95-96.
[10] V. S. Seleznev, E. O. Antonova. Implementation on Sobel Field-
Programmable Gate Array Detector for Identification of Vehicles.
Conference Proceedings: 2021 Intelligent Technologies and Electronic
Devices in Vehicle and Road Transport Complex (TIRVED).

Authorized licensed use limited to: Odisha University of Technology and Research. Downloaded on August 29,2024 at 15:24:03 UTC from IEEE Xplore. Restrictions apply.