AUTCON-01614; No of Pages 15

Automation in Construction xxx (2013) xxx–xxx

Contents lists available at SciVerse ScienceDirect

Automation in Construction
journal homepage: www.elsevier.com/locate/autcon

Image-based retrieval of concrete crack properties for bridge inspection

R.S. Adhikari ⁎, O. Moselhi 1, A. Bagchi 2
Department of Building, Civil, and Environment Engineering, Concordia University, Montreal H3G 1M8, Canada

a r t i c l e i n f o a b s t r a c t

Article history: Cracking can invite sudden failures of concrete structures. The objective of this research is to develop an in-
Accepted 20 June 2013 tegrated model based on digital image processing in developing the numerical representation of defects. The
Available online xxxx integration model consists of crack quantification, change detection, neural networks, and 3D visualization
models to visualize the defects in such a way that it mimics the on-site visual inspections. The crack quanti-
Keywords:
fication model evaluates crack lengths based on the perimeter of the skeleton of a crack which considers the
Crack quantification
Visual inspection
tortuosity of the crack. The change detection model is based on the Fourier Transform of digital images elim-
Change detection inating the need for image registration as required in the traditional. Also, the integrated model as proposed
Spectral analysis here for crack length and change detection is supported by neural networks to predict crack depth and 3D
Branch points visualization of crack patterns considering crack density as a key attribute.
Neural networks © 2013 Elsevier B.V. All rights reserved.
3D visualization

1. Introduction in every two years. The major tasks outlined in bridge inspection man-
uals during routine bridge inspection are two folds [7,8]. The primary
The problems of cracking in reinforcement concrete structures purpose of the routine bridge inspection is to inspect the physical con-
are natural and can invite spectacular failure of entire structures [1]. dition of bridge structures in terms of the extent and severity of defects
There always exist constraints in reinforced concrete structures and (quantification model) and the secondary purpose is to verify and up-
hence cracking is unavoidable regardless of the types of structures. date information about structures reported in last inspections (change
Cracks not only provide access to harmful and corrosive chemicals detection model). Traditionally, the routine bridge inspection is carried
inside concrete, but also allow water and de-icing salts to penetrate out through visual inspection and several limitations of visual inspec-
through bridge decks which can damage superstructures and bridge tion have been identified by previous researchers [4,5,10,36]. For exam-
esthetics [2]. In spite of significant research in addressing the prob- ple, it provides only qualitative information of defects as well as the
lems of cracking in bridges, it still remains a challenging problem inspection process is laborious, time-consuming and influenced by sub-
whether it is old and newly constructed bridges [3]. Therefore, a rig- jective behavior of individual inspectors. The aim of this research is to
orous study towards the evaluation of the extent and severity of enhance the two fundamental tasks of routine bridge inspection consid-
cracks is necessary for condition assessment of bridges, and to main- ering crack as a major concrete defect for condition assessment of con-
tain database for bridge inspections for long-term analysis [3]. crete bridges. The authors propose an integrated model which develops
Currently, the method of data collection for bridge inspections is by numerical representation of concrete defects based on digital image
performing site visits [4,5]. The collected information during field visits processing to achieve the objective of this research. The proposed inte-
is the fundamental input data for automated Bridge Management Sys- gration model consists of crack quantification, change detection, neural
tems (BMSs), such as PONTIS [10]. In Canada, there is a need for a uni- networks, and 3D visualization models to represent concrete defects
fied BMS to maintain Canadian bridges. Canadian bridges need to be with few numbers and to visualize the defects in such a way that it
inspected frequently to ensure that the functionality of structural mem- mimics the on-site visual inspection.
bers meets the safety requirements because more than 40% of all brid- Many attempts have been made earlier to enhance traditional ap-
ges are older than 50 years [6]. In practice, the existing condition of proaches of condition assessment of concrete bridges. Abudayyeh et
bridges is documented through routine bridge inspections performed al. [66] proposed a framework for automated bridge imaging system
based on digital image processing and integrated with Bridge Manage-
ment Systems (BMSs) PONTIS. Their models were capable of storing dif-
⁎ Corresponding author. Tel.: +1 514 848 2424x7144; fax: +1 514 848 7965. ferent surface defects. However, the condition rating of bridge elements
E-mail addresses: [email protected], [email protected] was assigned manually displaying the collected images on computer
(R.S. Adhikari), [email protected] (O. Moselhi), [email protected]
(A. Bagchi).
monitors. To automate this process, an automated prediction of condi-
1
Tel.: +1 514 848 2424x3190. tion rating model is required. However, the pre-requisite to such auto-
2
Tel.: +1 514 848 2424x3213. mated models is the digital representation of defects i.e. the defects or

0926-5805/$ – see front matter © 2013 Elsevier B.V. All rights reserved.
http://dx.doi.org/10.1016/j.autcon.2013.06.011

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
2 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx

change detection that needs to be defined numerically to automate the visualization of crack density by projecting digital images and neural
process. Such numerical models can be trained for automated predic- network models to predict crack depth necessary for condition assess-
tion of condition state rating as proposed by Adhikari et al. [67]. ment of concrete components. With the successful application of the
Hence, this paper focuses on the development of an integration model proposed approach, the two tasks of routine bridge inspection can be
which can be integrated with an automated system to overcome the greatly enhanced by incorporating them in an automated Bridge Man-
limitation of traditional approach of condition assessments of bridges. agement System.
Several Non-Destructive Testing (NDT) techniques such as corro-
sion detection, alkali–silica reactions, and sulfate attacks exist to iden- 2. Background
tify the existence and extent of deterioration in concrete structures in
order to understand the behavior of concrete structures [9]. However, 2.1. Cracking patterns in bridge decks
the detection of the extent and severity of a defect inside a concrete
component is laborious, time-consuming, and often unreliable. But, The characteristics of critical cracks need to be understood in
the concrete structures show the combined effects of internal and ex- reinforced concrete structures because the misunderstanding of the
ternal defects in the form of surface cracks which can be tracked with nature of cracks and their crack patterns can result in a sudden fail-
digital images. Also, bridges operate usually under fatigue loads and ure of structures. The common forms of crack patterns experienced
the components of bridges show initial fatigue reactions in the form in reinforced concrete structures are longitudinal, transverse, and di-
of cracks on surfaces. For this reason, this work adopts concrete cracks agonal cracks; crack map; and random cracks [11]. Transverse cracks
as the major defect parameter for condition assessment of concrete are generally formed perpendicular to the longitudinal axis of a
elements. For the analysis of cracks, crack quantification and identifi- bridge deck under the transverse reinforced steel. Transverse cracks
cation of crack patterns are necessary to reveal the condition of brid- are typically full depth across sections and crack spacings are 3–10 ft
ges. In recent years, many algorithms for automated crack detection as observed along the length of a bridge deck [12]. Field survey requires
have been studied in literatures [36,39,40,51]. However, the existing identifying the location and orientation of such cracks. Similarly, the
methods have several limitations which add difficulty in automating cracks running parallel to the longitudinal axis are called longitudinal
the entire process. A recent work on crack length quantification as re- cracks which appear just above the longitudinal reinforced steels.
ported in [36] uses an object-oriented bounding box which also ig- These cracks follow the paths of reinforcing steel in structural elements
nores the crack tortuosity in calculating the length of a crack. From a [13]. Crack patterns also differ from one bridge type to another. Skewed
practical point of view, cracked length needs to be divided into several bridge decks are more prone to diagonal cracks as compared to straight
small segments and average width of each segment requires to be an- aligned bridges [14]. Another way of visualizing a crack pattern is Map
alyzed separately. The proposed crack quantification model evaluates Cracking. Improper curing and restraining volumetric change of con-
the length of a crack from the crack skeleton perimeter which con- crete are the primary causes of Map Cracking [11]. Many attentions
siders the tortuosity, segmentation and branching of the crack. The have given in design of concrete structures. However, a proper model-
existing approach for crack segmentation requires searching of crack ing and visualization of the crack patterns are lacking which can aid in
pixel connectivity, which needs input of at least one point to start the decision making process effectively.
searching of connected points and matching with orientation angle
to decide whether the pixel in question belongs to same group or 2.2. Deck performance against cracks
not [36]. If there exists multiple of crack skeletons (unconnected
crack regions) in a same image frame, the existing approach of seg- Transverse cracks accelerate the process of corrosion by allowing
mentation becomes time-consuming. This method could be improved moisture, oxygen, and chloride ions into concrete spaces [15]. For
by automatic detection of the branch points and segments. Additional- reinforced concrete beams and decks, crack widths of less than
ly, the numbers of branch points in themselves could serve as a good 0.3 mm (0.01 in.) have little effect on overall corrosion of reinforcing
indicator of condition rating of concrete elements. The increasing steel [16,17]. As the width of cracks is more than 0.3 mm, the chances
numbers of branch points could be an indicator of decreasing integrity of corrosion in reinforcing steel increase and it leads to spalling of
of structural elements. The proposed method addresses some of these concrete. The recommended clear cover is necessary to protect
issues as well. reinforcing steel. In many cases, concrete spalling is the consequence
Another contribution of the current paper comes from a proposed of inadequate cover of reinforcing steel [18]. Since cracks are progres-
change detection model which provides a numerical representation of sive in nature, timely inspection and evaluation of critical crack length,
crack images making it easy for comparison of temporal defects. The width, and density are necessary to prevent failure of structures. Bridge
change detection model proposed in this work is based on spectral anal- inspection manuals and guidelines have different way of handling
ysis of images obtained through Fast Fourier Transform of digital im- cracks in concrete structures. For example, PONTIS [10] uses a smart
ages. The traditional approach of change detection is based on image flag element 358 for condition assessment of surface cracks in bridge
subtraction and image rationing which requires image registration pro- decks. This element addresses the extent of cracking in four condition
cess [60]. The image registration is cumbersome process and often it is states on the basis of crack width and crack [20]. After the recognition
difficult to achieve the desired accuracy [63–65]. The proposed ap- of element condition states, the proper action plan is necessary to repair
proach for change detection does not require image registration process the cracks. Any concrete cracks wider than 0.9 mm (0.035″) require to
and has added advantage of capturing the surface information be maintained by contractors according to Colorado Department of
contained in the image texture. Transportation in USA (0.035″) [19].
Identification and detection of individual cracks are not sufficient in
understanding the structural behavior. The engineer needs to know 2.3. Current practices of crack survey
how the crack patterns change with time and where they lie on struc-
tural members. For this purpose, this work develops 3D visualization The crack survey described here is based on the protocol developed
model of crack patterns measured through crack density defined as at the University of Kansas as a part of Pooled Fund TPF — 5(051) Con-
total length of cracks divided by total surface area. Maksymowicz et struction of Crack-Free Concrete Bridge Decks [3,21]. The cracks consid-
al. [71] developed 3D visualization models of defects based on digital ered in this protocol are greater than 0.2 mm because this size of cracks
image processing. However, their models were supported by manual is visible with the naked eyes while bending at the waist. The initial
operations using advanced graphical editor software to model defects preparation of deck surfaces is required before carrying out crack sur-
from photos. The proposed integrated model is supported by 3D vey. The bridge decks might require cleaning with clear water so that

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx 3

cracks can be seen with the naked eye. Then, a grid of 5 ft by 5 ft is long crack into small crack segments and retrieving average length
marked on the deck surface to locate the position and dimension of and width for each segment. In current literature, crack segmentation
cracks. This survey data is plotted on CAD drawings and overlaid on has been proposed by searching of crack connectivity and matching
structural drawing. The obtained crack map can reveal important dis- with pixel orientations [36]. This approach needs input of at least one
tinction between structural and non-structural cracks on the deck sur- crack skeleton pixel to start searching of crack connectivity. Moreover,
face [3]. However, this procedure is time-consuming and expensive. the length of a crack is estimated from object-oriented bounding box
Hence, a fast and economical way of preparing a crack map using digital which ignores the tortuosity of cracks [36]. Yu et al. [35] estimated
image processing is sought for 3D visualization of cracks. crack properties of a concrete tunnel through a graph. However, this ap-
proach also requires manual specification of the start and end points of
2.4. Automated crack detection using digital images cracks.
Apart from developing a model for crack detection and quantifica-
Bridge decks require more attention than the rest of bridge compo- tion, it is necessary to study the impact of cracks on structural members.
nents because they are under direct contact of service loading and are For this purpose, different types of cracks in reinforced concrete mem-
exposed to severe environmental impacts [22]. In recent years, several bers such as, flexural, shear, and diagonal tension cracks are important
automated crack detection algorithms had been proposed in literatures to be identified in terms of their location and severity. For example,
for condition assessment of reinforced concrete structures. For example, Siegel and Gunatilake [37] proposed the propagation of crack depth
the performance of crack detection had been tested against Fast Haar by tracing an element edge from high to low scale. This application
Transform (FHT), Fast Fourier Transform, Sobel, and Canny algorithms. was carried for a remote visual inspection of an aircraft surface. Crack
It was found that FHT was more reliable than others [23]. For automated depth determination and its propagation are vital for reinforced struc-
crack detection approach, it is necessary to discriminate cracked sur- tures to assess their structural integrity. For automatic identification of
faces from not cracked surfaces. A Principal Component Analysis crack depth in concrete slab, wave propagation techniques were tested
(PCA) had been applied successfully for pattern recognition whether a by Song et al. [31]. Proper study is required to develop some data fitting
surface was cracked or not based on Euclidean Distance as similarity models which can predict crack depth from visual inspection data avail-
measures criterion [24]. able during routine bridge inspection.
The crack analysis is necessary for automatic repair and mainte- Image quality is highly dependent upon complex environmental
nance of concrete surfaces. Haas et al. [25] developed an automated and lighting conditions. Cracks are always difficult to distinguish
field prototype crack sealing system which required fully automated from their backgrounds due to the noise caused by irregular illumi-
crack detection, surface mapping, and control systems. In digital imag- nation, shadings and moisture present on concrete surfaces [39]. For
ing process, crack pixels need to be separated from their background. proper image visualization, the dynamic range of camera and light-
For segmentation of a crack from its background, a threshold is used ing conditions are important to be considered to have satisfactory
for extracting the crack boundary pixels. The threshold can be estimated results for objects quantification [40]. The author's previous work
from mean and standard deviation of gray-level images [26]. However, on image-based retrieval of concrete crack properties shows differ-
this method does not ensure proper crack connectivity. To solve this ent aspects of image enhancement before segmenting defects from
problem, the percolation-based image processing method was sug- image frames [41]. For a proper visualization of cracks, Jahanshahi
gested to correctly detect cracks on concrete surfaces [27]. The result and Masri [38] proposed a method for crack detection using 3D
of this approach indicated that the percolation-based method is effi- scene reconstruction from digital images. Crack segmentations
cient even for larger-size concrete surfaces. In a later development, were adjusted automatically based on the depth parameters
mathematical morphology was adopted in an image segmentation obtained from 3D scene reconstruction using images taken from dif-
method where images were partitioned based on either similarity or ferent fields of view. However, digital images are influenced by per-
discontinuity. Lyer and Sinha [29] adopted morphological filters with spective and parallax errors which distort the geometry of an object,
linear structuring elements for defect identification in sewer pipelines. and hence the quantification of objects will be error-prone. Hence, it
Several types of concrete defects such as cracks, holes, joints, and col- is recommended that as far as possible, the acquired images shall be
lapsed surfaces were identified based on image processing and mor- orthogonal or close to be orthogonal to reduce the quantification er-
phological segmentation approach which is necessary for automated rors [42].
classification of defects in sewer lines [30]. However, this approach In order to overcome the limitations of visual inspection, several at-
was not suitable to discriminate all kinds of defects in sewer lines. tempts have been proposed to automate the current bridge inspection
Therefore, for certain objects like joints and lateral displacements process. One of such attempts is to automatically retrieve the three di-
were classified by using shape or texture features such as roundness mensional (3D) as-built/as-is bridge information using remote sensing
and compactness [30]. techniques. Many software have been developed for the design of struc-
Statistical approach for objects classification occupies a wide plat- tural components, however, a little attention has been given on how to
form in digital image processing. Neural networks were used to classify model the concrete surface defects using the parameters such as the ex-
different objects segmented from video images for automated object tent, severity and intensity of defects. The simplest way to describe the
identification on construction sites [28]. Moselhi and Shehab-Eldeen geometry of a defect is to collect the measurement of the defect by in-
[32] worked with digital image processing techniques and neural net- spectors, but it is a time-consuming and costly task. Although the 3D
works to detect and classify various defects present in sewer pipelines. as-built/as-is bridge information is useful, the retrieval of such informa-
The content based methodology was suggested for automatic identifi- tion is a challenging task [68]. McRobbie et al. [69] investigated several
cation of shapes, objects and materials assisting in construction and off-the-shelf 3D software tools, such as MeshLab, Rhino, TrueSpace, and
maintenance of projects [33]. Also, to capture uncertainties in objects Phtosynth, and found that existing tools could not fully support the au-
classification, a neuron-fuzzy network was proposed for defects classifi- tomatic retrieval of 3D as-built/as-is bridge information. A lot of manual
cation in sewer pipes [34]. editing and correction work are still required, which makes the overall
The purpose of image processing is to extract area of interest from a information retrieval process labor-intensive and time-consuming [70].
given image frame. For example, the extraction of length and width of A photographic technique was demonstrated for 3D visualization of
crack is necessary for crack analysis of concrete members. So, proper bridge components in estimating the measurement of bridge defects
definition of crack length and width needs to be stated explicitly. For [71]. However, the authors added an additional tool to “Damage Assess-
practical purpose, average crack width is required to prepare a bid for ment Graphic Analysis — DAGA” for modeling defects from photo-
repair and maintenance purposes. This could be possible by dividing a graphs. The above system was equipped with an advanced graphical

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
4 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx

editor enabled with fast creation of 3D models of bridge components Data

and presentation of the results graphically. A quick 3D visualization Acquisition
model based on digital images and available software tools is required (Camera)
to assist the condition assessment process for concrete bridges.
Crack
3. Proposed methodology Segment 2
Previous Work
The developed methodology encompasses structured procedures
for image capturing, image projection, crack segmentation, change Crack Crack
Image Stitch
detection, and data fittings with a 3D visualization of crack patterns Segment 1 Segment 3
as shown in Fig. 1. A set of functions such as crack identification
and quantification were designed and developed based on the con- Image Pre-
cept of crack branching which allows dividing cracks into compo- Processing
nents and retrieving their corresponding properties. The developed
methodology also provides a capability to follow the crack trends
Prespective Corrected
or propagation by comparing images acquired at different periods Affine Transform
Distortion Image
of time. This temporal comparison of cracks helps to find the per-
centage of deterioration in each image frame which is useful infor-
mation for modern Bridge Management Systems. Additionally, a Crack
Detection
neural network model was developed that maps relationship be-
tween crack depth and crack width. The information obtained from
Current Work
such Non-Destructive Test (NDT) can be studied together with mate- Neural
Data Fitting
rial science (fracture process) and stress analysis (damage tolerance Networks
design) which may reveal the secrets of spectacular failure of bridges
in the past or can prevent such failures in the future. Some of the as- Change
pects of this methodology have been discussed in details here. Detection

3.1. Image stitching and pre-processing 3D Application

Visualization Crack Density
An image stitching algorithm developed by Brown and Lowe [48]
Fig. 1. Research methodology.
has been adopted here. The method is based on extraction of the invari-
ant features of images and matching them with new images. This algo-
rithm works on feature based registration which is invariant to rotation,
zooming, and illumination change in input images. The detail discussion images are to overlap to facilitate image stitching. The orthography
on the image stitching problem can be found in the authors' previous projection is defined mathematically by Eq. (2) [42] as
work [41] and further details in [36,48].
In this work, each photo frame must include either a natural or arti- X¼mX & y¼mY ð2Þ
ficial target for scale calibration to get pixel value in millimeter. After
image acquisition, pre-processing is required for proper image en-
hancement. Generally, there are two image enhancement methods: where, m is called the scaling factors. This projection is an affine trans-
spatial domain operation and frequency domain operation. Spatial do- formation in which relative geometry relationship of objects is
main operation is divided into three parts: point processing, histogram maintained. In practice, if we acquire images very close to the scene, it
based techniques and mask processing [49]. The adoption of particular can be referred as orthography projection and is suitable for digital
algorithm is usually application dependent. image analysis [42]. However, in bridge inspection, it is not always pos-
sible to acquire images close to the scene, and we may need some trans-
3.2. Imaging criteria for data acquisition formation matrix to solve this problem. In general, a transformation
matrix is a composition of rotations, translations, magnifications, and
A digital image is a 2D projection of a 3D real world object. The effec- shears as expressed by Eqs. (3) & (4) [42,47].
tiveness of digital image analysis highly depends upon how 3D to 2D
projections occurs. The image projections can be broadly classified in x‵ ¼ A11  x þ A12  y þ b1 ð3Þ
two parts: perspective and orthographic projections. The camera coor-
dinate in perspective projection can be defined by Eq. (1) [42]. y‵ ¼ A21  x þ A22  y þ b2 ð4Þ

where, x‵ and y‵ are the transformed coordinate of points (x, y); (b1, b2)
x ¼ f  X=Z & y ¼ f  Y=Z ð1Þ
relates to translation; (A11, A12, A21, A22) relates to rotational; (A11,
A22) relates to the scaling matrix; and (c1, c2) is the projection vector.
where, a 3D real world coordinate is represented by (X, Y, Z); a cam- For an affine transformation, all elements of the projection vector are
era coordinate is given by (x, y); and f represents the focal length of a equal to 0. To illustrate the perspective error, synthetic images of differ-
conventional camera as shown in Eq. (1). Scheffy et al. [45] corrected ent length and width were drawn and developed algorithms tested on
the orientation of original images by removing warp and skew before these images as shown in Fig. 2a and b. Fig. 2a was taken by rotating
tracing the cracks properties. Similarly, McRobbie [46] showed that a the image by 10°, and Fig. 2b was taken at 45° angle with horizontal
single pixel in an image taken perpendicularly to surface would rep- plane. The analysis showed that error increased significantly up to 40%
resent a smaller area than the same pixel in an image taken at an in oblique images.
angle. They also listed various imaging criteria such as the minimum Projective transformation can be considered as a tool to project im-
pixel resolution to be 1 pixel/mm, camera position, elevation and ages on 2D planes such that geometric relationships within the images
bearing are to be recorded while taking images, and successive are not destroyed. The complete work flow for the projective

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx 5

transformation technique as adapted to the present work has been D4 ðp; qÞ ¼ abs ðx−sÞ þ abs ðy−tÞ ð6Þ
shown in Fig. 3 [42].
The synthetic image displayed in Fig. 2b was corrected against per-
spective error and displayed in Fig. 4. The errors in width calculation
D8 ðp; qÞ ¼ max ½abs ðx−sÞ; abs ðy−tÞ: ð7Þ
from the corrected image are within an acceptable range from inspec-
tion aspect of concrete elements in comparison to visual inspection.
From skeleton image, the following cracks descriptors which can be
3.3. Discrete crack detection and analysis easily obtained are length of the segment, points of at least 2 branches,
points of at least 3 branches, and number of branch points, and width
Practically, it is not feasible to develop a generic algorithm to extract profile.
all properties from digital images [44]. There is always human interven-
tion at some point in automation process and an acceptable level of 3.3.2. Crack length and width
human intervention shall be defined based on accuracy, efficiency, The above approach divides a given crack in various segments
and repeatability [43]. Procedures of retrieval of crack properties are subtracting branch points from the crack skeleton. The estimation of
explained in Fig. 5. crack length is based on the perimeter of crack skeleton. Practically,
The detection and analysis of cracks start with loading images, ex- cracks in concrete elements have less number of pixels as compared
ploring red, green, and blue (RGB) color planes and segmenting by suit- to their background and have large length to width ratios. Hence, half
able edge detection algorithms. of the perimeter of crack skeleton is a good approximation of crack
length. The area of a crack object in binary images is found by summing
3.3.1. Skeletonization of cracks area of each pixel. After getting the length and area of a crack object, its
Skeletonization is an important step to retrieve crack segments average width can be calculated by dividing total area by length of the
which show different paths at their branching points. This can be de- crack segment. The process has been described in Fig. 6e. Then each
scribed mathematically by distance measure in image processing do- crack segment is labeled and retrieved crack properties are retrieved.
main [49]. In digital domain, the algorithms for distance measures are However, additional criteria need to be defined to clean up noise pres-
Euclidean Distance, D4 distance or City Block Distance or Manhattan ent at the final stage to discriminate among cracks [51]. For example,
Distance, and Chase Board Distance. Considering three points p (x, y), the additional criterion for length estimation is that a crack must not
q (s, t), and z (u, v), the Euclidean Distance, D4 distance, and D8 distance have a length smaller than 20 pixels. This limit is application dependent
can be obtained by Eqs. (5), (6), and (7) respectively [49]. and can vary from one problem to another. The developed algorithms
are tested on real images shown in Fig. 6. The images in Fig. 6a, b, c,
h i and d show crack detection, branch points, estimation of crack length
2 2 §
De ðp; qÞ ¼ ðx−sÞ þ ðy−tÞ ð5Þ
and width, respectively.

Fig. 2. a — Images rotated by 10°. b — Images at 45° (oblique view).

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
6 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx

Interactively define co- Read & Display Explore RGB Color

Read image to be Read base Segmentation
ordinates of input Images Space
projected Reference Image
Image

Branch Points Skelaton Dilation

Create Projective Interactively define
Apply Projective
Transformation co-ordinates of base
Transform Modify Definition Extract Crack Set Object Label &
Structure Image
(e.g. length, Width) Properties Scale Factor

Crop and Display

Result
Corrected Image Visualization

Fig. 3. Projective transform work flow. Fig. 5. Crack detection algorithms.

3.3.3. Images from lab experiment crack depth and width (width in mm *100, to plot in the same graph)
Due to great variability in observing the crack depth and crack of a particular flexural crack under various loading stages was summa-
width, the accurate predictions of structural member's behavior is rized in Fig. 9 and Table 1.
possible only at a statistical level [53]. The experimental setup is
shown in Fig. 7 where a number of digital images were collected for 3.3.4. Data fitting for crack depth vs. crack width
data analysis. The details of cross sections and reinforcements are Masri [54] showed a method to predict the levels of damage by
shown in Fig. 7a. When tensile strength of concrete members reaches training neural networks from vibration measurements of struc-
its maximum allowable value, the cracking occurs in flexural members, tures. A back-propagation neural network and computational me-
and there is sudden loss of stiffness. These cracks are called primary chanics approach were used by Liu et al. [55] to classify type,
cracks which penetrate spontaneously to a certain depth. It was ob- location, and the length of cracks. Similarly, Xu and Humar [56]
served that the height of the primary cracks immediately after cracking used the modal energy-based damage index to determine location
was significant, and remained constant under increasing loads to some of damages and an ANN to determine the extent of damages in struc-
extent. However, the width of cracks increased gradually with load and tures. In this paper, a neural network was trained to predict depth of
time. The images were collected during the experiment by holding the crack given the width of a crack. A supervised neural networks model
certain value of load magnitude to collect the digital images. was proposed for this work having input attribute as the crack width
For crack detection, a camera was focused at the critical cracks just and output attribute as the crack depth. One hidden layer was used
below the points of application of loads, while another camera was with 10 neurons. A total of 101 crack patterns were obtained from
used to take different images covering the whole beam crack pat- the digital images to train the networks with 39 patterns defined as
terns. At various stages, the images were retrieved and their length the testing and validation sets. The training algorithms work flow
and width were calculated separately. To check the accuracy of chart for this network is explained in Fig. 10.
crack width obtained from the developed algorithms, a crack scale After defining the training, testing, and validation sets for the net-
shown in Fig. 8 was used to measure the actual crack widths. For illus- work, the actual training of the neural network was started. The
tration, the original image was rotated to 90° to fit the paper size and training stopped at 12th epochs since the validation error was con-
space. The obtained crack width using developed algorithms at stant after 6th epochs as shown in Fig. 12. The fitted data of the
115 KN load was found 0.89 mm as shown in Fig. 8a and b. The actual rest of the patterns were displayed in Fig. 11 with the distribution
width measured was 0.9 mm in Fig. 8. of error at each point.
American Concrete Institute (ACI) [51] and Ontario Structure In- Next the trained networks were tested with test data to evaluate the
spection Manual (OSIM) [61] assign permissible limit of crack width performance of the model as shown in Fig. 13. This figure shows data
in various weather exposure conditions as shown in Tables 2 & 3. fitted with 39 data patterns which were not exposed to network during
These limits are important parameters for condition assessments of training of network. The predicted data points and test data points
reinforced concrete structures. The result of tracking of the cumulative matched with acceptable degree of accuracy as shown in residual and

Fig. 4. Image after projective transformation (corrected image).

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx 7

Fig. 6. Original crack image, a — crack detection. b — Branch points, c — crack length. d — Crack width. e — Estimation of crack length and width.

Fig. 7. Experimental setup for beam loading under fatigue loads. a — Cross-section and Fig. 8. Use of crack scale for width measurement. a — Crack edge detection. b — Crack
reinforcement details. width.

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
8 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx

Table 1 Table 3
Evaluation of crack width and depth. OSIM [61], crack classification for reinforced concrete structures.

Image no. 0045.tif 0046.tif 0050.tif 0083.tif 0092.tif 00100.tif Hairlines cracks — less than 0.1 mm wide
Load in KN 30.00 50.00 70.00 90.00 100.00 115.00 Narrow cracks — 0.1 mm to 0.3 mm wide
Width of crack in mm 0.30 0.37 0.58 0.75 0.86 0.89 Medium cracks — 0.3 mm to 1 mm wide
Crack scale width in 0.23 0.30 0.50 0.77 0.85 0.90 Wide cracks — greater than 1 mm wide
mm
% error 30.43 23.33 16.00 −2.60 1.18 1.11
Depth of crack in mm 67.00 120.00 120.00 136.00 145.00 163.00
The result of the above equations was a pair of coordinates [S(r), S
(θ)]. The plot of these coordinates provided the information on the var-
histogram plot. The residual error was calculated using Eq. (8) [50]: iation in the spectral energy contained in the image texture. Typical de-
scriptors could be retrieved from the functions S(r) and S (θ) which
ErrorXPercentage ¼ ðabs ðResidualsÞ:=TestXDataÞ  100 ð8Þ could characterize their behavior quantitatively. Typical descriptors in-
cluded location of the highest value, the mean and variance of both the
where, Residuals = (Forecast_Data) − (Test_Data). amplitude and axial variations, and distance between mean and the
The plot of the residual indicated that the distribution of data was highest value of the function as shown in Fig. 16a & b.
very close to zero line. Most of the data points were distributed with Fig. 16b showed the plot of two cracks displayed in Figs. 15 and
+5 to −5 which was also clearly seen in residual histogram plot at 16a. The distribution of energy in radial direction between these
the top right of Fig. 13. two cracks was seen different which varies from 100 to 150 and the
The above trained model was compared with the Boosted Decision texture variation in angular direction showed a significant difference
Tree algorithms and the results are presented in Fig. 14. The result in peak values from 1350 to 1600 as illustrated in Fig. 16b. These dig-
showed that predicted data and test data were not fitted well, and the ital values were the numerical representation of digital images with
residuals plots were very scattered from +90 to −50 as shown in the few numbers which could be used as features to compare the extents
bottom of Fig. 14. This comparison showed that the model developed of the damage at different times.
by neural networks was superior as compared to the Boosted Decision
Tree models. 3.4.1. Use of median filter for temporal change detection
To monitor the change in crack patterns, the digital images of the
3.4. Distributed model for change detection same objects are required at different periods of time. For illustration
purpose, the mask processing techniques were used to smooth images
Over the past few years, several Electrochemical Fatigue Sensor to get different extent of damages. The general operation of mask pro-
(EFS) devices were developed to detect flows in metal. However, cessing could be defined by Eq. (11) [47].
such applications to detect the changes in damage patterns in
reinforced concrete structures are challenging, labor-intensive, and Gðx; yÞ ¼ T ½f ðx; yÞ ð11Þ
expensive. The present research proposes a new method to detect
change in cracks patterns over time by analyzing the digital images
where, T: a transformation function which operates on the original
in the frequency domain. Fig. 15 illustrates the work flow diagram
image f(x, y) to produce a modified image G(x, y). The size of the
for change detection procedure in spectral domain.
mask considered as [3 × 3], [5 × 5], or [7 × 7] depending upon the de-
Spectral descriptors can provide quantitative information of images
gree of roughness or smoothness required. The first column of Fig. 17
taken at different times to classify and rank them. For this operation, the
showed the same images at different damage levels using different
original images were converted into frequency domain by Fast Fourier
masks, the second column showed the plot of the radial energy varia-
Transform (FFT) as shown in Fig. 16. For convenience, Fourier Spectrum
tion, and the third column showed the angular variation of texture
was converted to polar coordinates. This procedure yielded a function S
contained in the images. The statistical values were summarized in
(r, θ) called spectrum function, where r and θ were the variables in polar
the Implementation and result section.
coordinate system [47]. Two important relationships as given by
Eqs. (9) and (10) were derived from the frequency plot.
3.5. 3D visualization and crack density
A global description obtained by integrating these functions as
shown in Eqs. (9) & (10) [47]:
Alan [57] proposed a new framework for object recognition process
h i using BIM. This system supported the storage and retrieval of objects of
SðrÞ ¼ Sum SðθÞ ðrÞ ð9Þ
interests in his model. Ryan [58] showed his work in evaluating the

where, θ varies from 0 to π.

h i 180.00 Plot of Depth & Width Vs Loads
Cum. Depth & Width in mm

S ðθÞ ¼ Sum SðrÞ ðθÞ ð10Þ 160.00

140.00
where, (r) varied from 1 to R0/2, R0 = the radius of a circle centered 120.00
at the origin. 100.00
80.00
60.00
Table 2 40.00
ACI 224R-01 [52] permissible crack width in reinforced concrete structures.
20.00
Exposure condition Maximum allowable width 0.00
20.00 40.00 60.00 80.00 100.00 120.00 140.00
Dry air 0.016 in (0.406 mm)
Humidity, moist air, soil 0.012 in (0.305 mm) Load Applied in KN
Deicing chemicals 0.007 in (0.178 mm)
Total Crack Depth Crack Width
Sea water 0.006 in (0.152 mm)
Water retaining structures 0.004 in (0.101 mm)
Fig. 9. Plot of Cum. crack depth and width.

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx 9

Input
Define Input/
Attributes Import File
Output Attibutes

ANN
Test And
Set Stopping Architecture
Validation Set
Criteria’s Design
Extraction

Application
Learning output of trained
networks

Fig. 10. Neural network training flow chart.

surface defect detection capability of terrestrial light detection and

ranging (LiDAR) for reinforced concrete bridge decks by generating a
3D model from point clouds. The above approach required point clouds Fig. 12. Error in train, validation, and test sets.

to be matched with digital images to identify the members and defects

on elements. In this paper, the 3D visualization was developed by
projecting digital images as well as the image texture. For better 3D For visualization, the Google Sketch-up (software for 3D visualization)
models, 4 images were required at each of the four sides of an object. [59] was used to develop the 3D models from digital images.
However, a 3D model can be developed just from a single image taken This work presented an integrated model which could be used to en-
at 45° so that at least three corners were visible from a single position. hance the two important tasks of routine bridge inspection. The inte-
The results were displayed in Fig. 18a and b which develop 3D grated model consisted of crack quantification model, image correction
models of a beam showing cracks on the front face and back face, re- model, change detection model, neural networks model, and 3D visual-
spectively. Based on this model, specific parameters such as crack den- ization models. This work demonstrated numerical representation of
sities were calculated to differentiate the severity of cracks and cracking defects which could be combined with the author's previous work [67]
patterns. These 3D models could be used to identify structural cracks vs. of neural networks models for automated prediction of condition state
non-structural cracks by comparing the crack patterns. rating of concrete members. The successful application of this approach
could overcome the limitations of information model (i.e. manual as-
signment of condition rating of bridge defects) developed by Abudayyeh
4. Implementation and result et al. [66]. Further, the proposed model could be merged with a Bridge
Management System (BMS) and the performance in decision making
Models for this work were developed in a Window Vista Enterprise process could be greatly enhanced with higher degree of reliability.
32 bit operating System. The desktop consisted of Intel® Core™ 2 Duo The present work successfully demonstrated the crack quantifica-
CPU, E6550 @ 2.33 GHz. The methodology for crack detection and quan- tion model. The work adopted a new approach of crack segmentation
tification algorithms was implemented in MATLAB R2012a [50]. The method based on key marks in an image frame called branch points,
tic-toc clock for training of neural network in MATLAB showed that crack length estimation based on crack skeleton perimeter which ac-
the elapsed time was 15.12 s. The implementation of these algorithms tually considered the tortuosity of crack instead of estimating crack
in C or C++ could improve computational efficiency to some extent. length from an object-oriented bounding box, and the average crack

Fig. 11. Data fitting by neural networks.

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
10 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx

Fig. 13. Performance evaluation of trained networks.

width considering the characteristics of the entire crack pixels instead crack width measured from a crack scale in Fig. 8. The results showed
of questing how many points need to be considered to obtain the av- that the developed algorithms worked well for crack width greater
erage crack width. than 0.3 mm, however, errors were not acceptable when the crack
First, this approach overcomes the limitation of the manual input of widths were less than 0.3 mm. Since the OSIM [61] and ACI [52]
at least one skeleton point to start searching of the crack connectivity to mention that for visual inspection purpose, a crack width less than
define crack segmentation boundaries [36] and is efficient even when 0.3 mm falls in a narrow class and may not be documented. So, the
multiple unconnected crack skeletons are present in the same image developed algorithms are suitable for inspection of cracks in
frame. Previously, the length of a crack was approximated from a reinforced concrete structures with the camera used in this work.
bounding box enclosing the crack skeleton. The maximum and mini- However, the accuracy of the crack width estimation can be im-
mum crack widths were successfully estimated in previous work [36]. proved by using high resolution cameras which may be able to detect
However, the average crack width estimation, which is important infor- crack width less than 0.3 mm.
mation required for the industry experts, has not been discussed. To Comparing the status of defects with previous inspection is an in-
demonstrate the proposed method, a set of 101 images were retrieved tegral part of routine bridge inspection. This work overcomes the
from the experimental study as illustrated in Fig. 7. A part of the result limitations of the existing methods as indicated by Singh [63]. For ex-
of the crack analysis performed over these images using the developed ample, he stressed the need to explore the possibility of developing a
algorithms was illustrated in Table 1. The depth of a crack represents change detection procedure that requires a less precise registration
the perpendicular distance from the top of a crack tip to the bottom of of images or simply bypass the registration process. The reason is
the beam surface. The width of cracks obtained at various loading stages that misregistration of images produces a number of false alarms
was displayed in Table 1 and the results were compared with the actual and often a precise registration is very difficult to obtain [63].

Fig. 14. Comparison with Boosted Decision Tree.

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx 11

to 48% by taking the images at 45° angles with horizontal plane (per-
Plot of
Original Image FFT spective error). In bridge inspection, it is not always possible to take
S ( r ) Vs r
images in orthogonal views and there need to apply geometric transfor-
mations to correct such errors. After applying orthogonal transforma-
tion, the error was reduced to below 15% on test images which is
Feature Plot of acceptable in many cases from bridge inspection purpose in comparison
Extraction S ( theta ) Vs Theta to visual inspection of bridges shown in Fig. 4.

5. Discussion and future work

Fig. 15. Change detection work flow diagram.

The reliability of visual inspection is a widely discussed topic in

asset management of civil infrastructures. In spite of several limita-
Hence, the proposed methodology provides a fast and easy way to tions of visual inspection, it is one of the widely accepted methodolo-
detect changes since image registration is not required and provides gies used in practice for asset management for buildings and bridges.
a few numerical representations of defects. This paper demonstrated Currently, due to advancement in computer technology, researchers
the change detection techniques in spectral domain as opposed to are working towards automating the inspection process using digital
the traditional spatial domain and the results were summarized in image analysis. The models developed through this process need to
Table 4. The difference between any two successive points either in be integrated with Bridge Management Systems (BMSs) to enhance
peak or mean domains illustrated the relative change in damages the reliability of decision making process. One of the important
present in each image taken at different times. This method could input information required in BMS is the condition rating of bridge el-
serve as a useful tool to enhance the current practice of visual inspec- ements. This paper proposed and demonstrated an integrated model
tion since it considered the texture information of concrete surfaces for the numerical representation of defects which was pre-requisite
as well. information required for developing automated predicted condition
Another important aspect of this paper was to display the infor- rating for bridge elements. The preliminary work for the automated
mation in the form of a 3D model. Based on the digital images, 3D prediction of condition rating of bridge elements had been demon-
models were developed as shown in Fig. 18. These crack patterns in strated in the authors' previous work [67]. An extensive research is
Fig. 18 were observed under the load of 115 KN. The model was de- required in developing several condition state rating models for
veloped by projecting the images by finding the matching lines in x each type of defects in concrete elements. So far, this paper discussed
and y directions. Crack density could be visualized from these models an integrated model based on cracks only.
which showed crack density of 0.0066 mm/mm2 at the front face, Cracking in concrete can cause failure of bridges due to fracture of
0.0058 mm/mm2 at the back face, and 0.0085 mm/mm2 at the bottom critical elements, and the following two aspects need to be understood
face. The other supported model for condition assessment demonstrat- to deal with these phenomenons [72]. First, a mechanism is required to
ed in this work was the neural networks based model to predict crack capture cracks with the aid of Non Destructive Test (NDT) to access the
depth. The crack patterns are important to identify during a bridge in- current condition of elements. Second, a failure assessment of bridge
spection which includes the types of cracks, location of cracks, crack components needs to be evaluated based on element fracture mecha-
width and depth, and crack spacing. In this paper, a neural networks nism. The first criterion has been discussed to a great extent in this
model was trained to predict the depth of cracks given the widths of paper to capture crack and its quantification using digital image process.
known cracks. This information would be important for evaluating the However, the estimation of the remaining life of a component based on
integrity of structural members as well as making decisions on repair the fracture defects needs further study. The neural networks model
and maintenance. The trained networks performance was acceptable discussed in this work can serve as the starting point towards this
with the Mean Absolute Percent Error (MAPE) to be equal to 1.66%. study. More tests are required to capture the effects of different mate-
The plot of residuals showed a normal distribution centered on zero. rials and their strengths which would ultimately affect the relationship
The data fitting algorithms were tested with the Boosted Decision between the crack depth and width. In the future, the authors propose
Tree method and the Mean Absolute Percent Error (MAPE) was found to develop finite element models to study the effect of fracture mecha-
to be 24.88%. Also, the residuals in this case were wide spread as nism of concrete elements under cyclic loads. From such simulations, it
shown in Fig. 14. The analysis showed that the neural networks data fit- may be possible to establish a relationship between time and the width
tings were superior over the Boosted Decision Tree model. of cracks for example. In short, future study in that direction will be able
The 3D visualization of cracks was demonstrated in the Google to answer the question of what will be the revised inspection frequency
Sketch-up software in displaying the crack density of concrete element for an element or what is the remaining life of an element by combining
under cyclic loads. In the past, researchers used several commercial fracture mechanics with NDT.
softwares to develop 3D visualization model such as MeshLab, Rhino, This work highlights the importance of orthogonal projections. Since
TrueSpace, and Photosynth [69]. However, this kind of software be- perspective errors destroy geometry of objects, any quantification of de-
comes obsolete after a few years. Some authors have used advanced fects will not be correct thereafter. Commercial softwares are available
graphical software to model defects and their quantification for bridge to correct such errors, but the use of different software makes the auto-
inspection purpose [71]. Hence, easily accessible and comprehensible mation process difficult and human intervention is mandatory to some
model sought to be developed for proper visualization of defects. This extent. Another problem is quantification of defects in common units of
work adopted the Google Sketch-up software [59] which is commer- measurements. Previous researches demonstrated crack width mea-
cially available and has the capability to project texture data from digital surement in pixels not in actual unit of measurement of defects such
images directly on the models to simulate the on-site visual inspection as in mm or inches [36]. It is evident that number of pixels is different
process. This approach does not require a field measurement survey for if we take images of same cracks at different field of view. Although,
3D visualization [62]. this problem has been addressed by Jahanshahi and Masri [38], the pro-
Section 3.2 of this paper presented how an image can be retrieved cess needs to be automated with the application of Ground Positioning
to achieve accurate quantification of defects. The test done on several System (GPS).
synthetic images showed that orthogonal images were required for The developed 3D crack models based on digital images provide
the image analysis because they contained the least perspective additional information about severity of cracks. The model developed
and parallax errors. The result showed that the error increased up for 3D visualization was based on the Google Sketch-up software.

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
12 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx

Fig. 16. Fourier Transform of digital images at time T1. a. Fourier Transform, radial, and angular plot of digital images at time T2. b. Comparison of descriptors at time T1 and T2.

Although this method was efficient in photo projection with texture, the work done by Zhu et al. [73] of detection of large-scale column
human intervention was required for drawing 3D elements on screen. for bridge inspection and the preliminary work has been accepted
The authors are working to automate this approach combining with for automated bridge condition assessment with hybrid sensing [74].

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx 13

Fig. 17. Temporal change detection.

Fig. 18. a — 3D visualization, (front face crack density = 0.0066 mm/mm2). b — 3D visualization, (back face crack density = 0.0058 mm/mm2, bottom face crack density =
0.0085 mm/mm2).

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
14 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx

Table 4 elements. Later, the developed models can be updated when further
Extraction of descriptors from spectral analysis. information is available either from lab tests or field investigations.
Images descriptions Radial change Angular change Finally, this paper highlighted the importance of 3D visualization
model which mimicked the on-site visual inspection. The visualiza-
Original crack (6)
Peak 2052 1372 tion models were demonstrated and the importance of crack density
Mean 1537 1167 required for industry experts. The proposed integrated model could
be merged with a BMS to enhance the reliability of decision making
Smoothed image (5 pixels) (5)
Peak 2052 1236 process for the condition assessment of concrete bridges.
Mean 1333 1017

Smoothed image (10 pixels) (4) Acknowledgment

Peak 2048 1221
Mean 1157 885 The authors would like to thank Concordia University, Montreal,
Smoothed image (15 pixels) (3) Canada; and Natural Sciences and Engineering Research Council of
Peak 2037 1214 Canada (NSERC) for financial supports for this research. The valuable
Mean 1008 777 comments and information about condition of bridges in Quebec pro-
Smoothed image (20 pixels) (2) vided by Mr. Adel Zaki, Chief Engineer (Roads and Bridges), SNC Lavalin
Peak 2029 1207 Inc. are also gratefully acknowledged. Thanks are also due to Mr. Arash
Mean 958 760 Rahmatian, a PhD candidate at Concordia University, for providing digi-
Smoothed image (100 pixels) (1) tal images from a fatigue test conducted on FRP-reinforced concrete
Peak 1826 1165 beams. The contribution of all experts participated in this work is also
Mean 865 671 gratefully acknowledged.

References
6. Conclusion
[1] S.H. Rizkalla, M.El. Shahawi, C.K. Kwok, Cracking behaviour of reinforced concrete
members, Canadian Society for Civil Engineering 1982 Annual Conference, Edmonton,
Visual cracks are frequently a concern for inspection engineers be-
Alberta May 27–28, 1982.
cause they provide easy access for harmful chemicals into concrete [2] P.D. Krauss, E.A. Rogalla, Transverse Cracking in Newly Constructed Bridge Decks,
which affect the integrity of structural systems. This paper proposed NCHRP Report 380, Transportation Research Board, National Research Council,
Washington, D. C, 1996.
an integrated model based on digital image processing to enhance the
[3] S. Ganapuram, M. Adams, A. Patnaik, Quantification of Cracks in Concrete Bridge Decks
fundamental tasks of routine bridge inspections. The integrated model in Ohio District 3, FHWA/OH-2012/3, Ohio Department of Transportation, 2012.
comprised of crack quantification, change detection, neural networks, [4] Federal Highway Administration (FHWA), Bridge Inspections Training Manual,
and 3D visualization models for describing concrete components de- July 1991.
[5] Bridge Maintenance Training Manual, US Federal Highway Administration,
fects by a few numerical numbers. Since bridges are prone to fatigue FHWAHI-94-034. , Prepared by Wilbur Smith Associates, 1992.
loading under service conditions and cracks are the initial indicators [6] L.A. Bisby, M.B. Briglio, ISIS Canada Educational Module No. 5: An Introduction to
for concrete fracture, this work develops several models considering Structural Health Monitoring. Prepared by ISIS Canada, 2004. 3.
[7] U.S. Army Corps of Engineers, Bridge Inspection Program FY 93 Routine Inspec-
cracks as concrete defects. However, the numerical representation of tions, New England Division, 424 Trapelo Road, Waltham, MA, 02254-9149, 1993.
concrete defects (quantification model) is largely dependent on camera [8] Navy Bridge Inspection Program on “Bridge Inspection and Reporting Guide-
resolution and imaging criteria. This paper highlights the importance of lines”. prepared by Collins Engineers, Inc., August 2008
[9] Ministry of Transportation, Ontario structure inspection manual, Policy, Planning
orthogonal images and presents algorithms to correct perspective & Standards Division, Engineering Standard Branch, October, 2000, (Revised on
errors. Nov. 2003, April 2008).
The existing crack segmentation method was based on searching [10] R.M. Gutkowski, N.D. Arenella, PONTIS, Investigation of Pontis — A bridge Manage-
ment Software, 1998.
of crack connectivity and required input of at least one crack skeleton
[11] T.R. Schmitt, D. Darwin, Cracking in Concrete Bridge Decks, SM Report No. 39,
point to start searching of crack connectivity. The proposed method University of Kansas Center for Research, Inc., Lawrence, 1995. (152 pp.).
detected branch points (key marks) in a given image frame and [12] C.W. French, L.J. Eppers, Q. Le, J.F. Hajjar, Transverse Cracking in Concrete Bridge
Decks, Transportation Research Record, No. 1688, Proceedings of the 78th Annual
crack segments were automatically defined by subtracting branch
Meeting, January 10-14, 1999, Washington, D.C, Transportation Research Board,
points from the crack skeleton. This approach avoided searching of Washington, D. C., 1999, pp. 1–29.
crack connectivity and would be efficient in terms of time and cost. [13] R.H. Curtis, H. White, NYSDOT Bridge Deck Task Force Evaluation of Bridge Deck
Further, crack length was estimated from the crack skeleton perime- Cracking on NYSDOT Bridges, New York State Department of Transportation,
February 2007. 1–26.
ter. Since the aspect ratio of crack length to crack width is large and [14] G. Fu, J. Feng, J. Dimaria, Y. Zhuang, Bridge deck corner cracking on skewed struc-
crack skeleton is a single pixel connected path, this approximation tures, Department of Civil and Environmental Engineering, Wayne State Univer-
is reasonable in evaluating the length of a crack. . In recent literature, sity, Research Report RC-1490, September 2007, pp. 1–153.
[15] NCHRP Synthesis 333, Concrete Bridge Deck Performance — a Synthesis of High-
the crack length was estimated from object-oriented bounding box way Practice, Transportation Research Board, 2004. , (87 pages).
which ignored the tortuosity of the crack. [16] J.T. Houston, E. Atimay, P.M. Ferguson, Corrosion of Reinforcing Steel Embedded
The tradition method of change detection was based on mainly in Structural Concrete, Re-port CFHR-3-5-68-112-1F, and Center for Highway Re-
search, University of Texas, Austin, 1972.
image subtraction and image rationing which required images to be [17] J. Ryell, B.S. Richardson, Cracks in Concrete Bridge Decks and Their Contribution
registered first. Image registration is time-consuming process and to Corrosion of Reinforcing Steel and Prestressing Cables, Report IR51, Ontario
often difficult to achieve the desired accuracy in spite of development Ministry of Transportation and Communications, Downsview, ON, Canada, 1972.
[18] R.E. Carrier, P.D. Cady, Deterioration of 249 bridge decks, Highway Research
of advance algorithms. The proposed approach completely avoids
Board, No. 423, 1973. 46–55.
image registration by analyzing images in frequency domain and the [19] X. Yunping, S. Benson, Abu-Hejleh Naser, A. Andi, A. Suwito, X. Zhaohui, et al., As-
numerical values of change detection have been obtained from Fourier sessment of the Cracking Problems in Newly Constructed Bridge Decks in Colorado,
Department of Transportation — Research, Colorado, 2003.
Transform of digital images. The numbers obtained from this model
[20] Bridge inspection manual, Minnesota Department of Transportation, Version 1.8,
make the automation process possible to compare the temporal images. October, 2009.
This paper also developed neural networks models to predict [21] Pooled Fund Project Crack Survey Protocol, Pooled Fund TPF-5(051) Construction
crack depths given crack widths. Since neural networks have advan- of Crack-Free Concrete Bridge Decks, Kansas Department of Transportation, 2011.
[22] J.H. Nam, S.J. Yoon, H.D. Moon, D.M. OK, S.J. Hong, Development of FRP-Concrete
tages to update models when additional information is available, Composite Bridge Deck in Korea — State-of-the-Art Review, Key Engineering Ma-
they serve as a useful tool for condition assessment of concrete terials 326–328 (2006) 1705.

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011
 R.S. Adhikari et al. / Automation in Construction xxx (2013) xxx–xxx 15

[23] I. Abdel-Qader, O. Abudayyeh, M.E. Kelly, Analysis of edge-detection techniques [47] R.C Gonzalez, R.E. Woods, S.L. Eddins, Digital Image Processing using Matlab, 2nd
for crack identification in bridges, Journal of Computing in Civil Engineering 17 Edition, 2012. ISBN 978-0-9820854-0-0. 2002.
(4) (2003) 255–263. [48] M. Brown, D. Lowe, Automatic panoramic image stitching using invariant fea-
[24] I. Abdel-Qader, S. Pashaie-Rad, O. Abudayyeh, S. Yehia, PCA-based algorithm for tures, International Journal of Computer Vision 74 (1) (2007) 59–73.
unsupervised bridge crack detection, Advances in Engineering Software 37 (12) [49] Online Video Lectures by Prof. P.K. Biswas, IIT Kharagpur, Department of
(2006) 771–778. Electronics and Communication Engineering, http://nptel.iitm.ac.in/syllabus/syllabus.
[25] C. Haas, C. Hendrickson, S. McNeil, D. Bullock, A field prototype of a robotic pave- php?subjectId=117105079.
ment crack sealing, The 9th International Symposium on Automation and Robotic [50] MATLAB version R2012a, The Technical Computing Computer Software, the Math
in Construction, June 3–5, Tokyo, Japan, 1992. Works Inc., 2012
[26] H. Cheng, X. Shi, C. Glazier, Real-time image thresholding based on sample space [51] L. Richard, C. Xuemin, W. Min, Automatic Crack Monitoring System, Texas DOT
reduction and interpolation approach, Journal of Computing in Civil Engineering 7-3997, Texas Department of Transportation, 2001.
17 (4) (2003) 264–272. [52] ACI 224R-01, Control of cracking of concrete structures, ACI Committee 224,
[27] T. Yamaguchi, S. Hashimoto, Fast crack detection method for large-size concrete American Concrete Institute, Farmington Hills, MI, 2008.
surface images using percolation-based image processing, Machine Vision and [53] R.I. Gilbert, Shrinkage cracking in fully restrained concrete members, ACI Struc-
Applications 11 (5) (2009) 797–809. tural Journal 89 (2) (1992) 141–149.
[28] Seokho Chi, Carlos H. Caldas, Automated object identification using optical video [54] S.F. Masri, Application of neural networks for detection of changes in nonlinear
cameras on construction sites, Computer-Aided Civil and Infrastructure Engineer- systems, Journal of Engineering Mechanics 126 (2000) 666–676.
ing 26 (2011) 368–380. [55] S.W. Liu, J.C. Sung, C.C. Lee, J.H. Huang, Detection of cracks using neural networks
[29] S. Iyer, S.K. Sinha, Segmentation of pipe for crack detection in buried sewers, and computational mechanics, Computer Methods Applied Mechanics and Engi-
Computer-Aided Civil and Infrastructure Engineering 21 (2006) 395–410. neering 191 (2002) 2831–2845.
[30] Sinha, Fieguth, Segmentation of buried concrete pipe images, Automation in Con- [56] H. Xu, J. Humar, Damage detection in a girder bridge by artificial neural network tech-
struction 15 (2006) 47–57. nique, Computer-Aided Civil and Infrastructure Engineering 21 (2006) 450–464.
[31] W. Song, J.S. Popovics, J.D. Achenbach, Crack depth determination in concrete [57] M.L. Alan, A Framework for Object Recognition in Construction Using Building In-
slabs using wave propagation measurements, Proceedings of the 1999 DAA Air- formation Modeling and High Frame Rate 3D Imaging. (PhD Thesis) Virginia Poly-
port Technology Transfer Conference, Atlantic City, 1999. technic Institute and State University, 2011.
[32] O. Moselhi, T. Shehab-Eldeen, Classification of defects in sewer pipes using neural [58] Ryan C. Hoensheid, An Evaluation of Surface Defects Detection in Reinforcement
networks, Journal of Infrastructure Systems 6 (3) (2000) 97–104. Concrete Bridge Decks using LiDAR, (Master thesis) Michigan Technological Uni-
[33] I.K. Brilakis, L. Soibelman, Y. Shinagawa, Construction site image retrieval based on versity, 2012.
material cluster recognition, Advance Engineering Informatics 20 (2006) 443–452. [59] Google sketch up, 3D for everyone, http://www.sketchup.com/intl/en/index.html.
[34] S.K. Sinha, P.W. Fieguth, M.A. Polak, Computer vision techniques for automatic [60] W. Guo, L. Soibelman, J.H. Garrett Jr., Automated defect detection for sewer pipe-
structural assessment of underground pipes, Computer-Aided Civil and Infra- line inspection and condition assessment, Automation in Construction 18 (2009)
structure Engineering 18 (2) (2003) 95–112. 587–596.
[35] S. Yu, J. Jang, C. Han, Auto inspection system using a mobile robot for detecting [61] Ontario Structure Inspection Manual (OSIM), The bridge inspection manual,
concrete cracks in a tunnel, Journal of Computation in Civil Engineering 17 (4) the ministry of transportation of Ontario, http://www.ogra.org/lib/db2file.
(2006) 255–263. asp?fileid=24271 2008.
[36] Z. Zhu, S. German, L. Brilakis, Visual retrieval of concrete crack properties for au- [62] S. Ganapuram, M. Adams, A. Patnaik, Quantification of Cracks in Concrete Bridge
tomated post-earthquake structural safety evaluation, Automation in Construc- Decks in Ohio District 3, The University of Akron, 402 Buchtel Common, Akron,
tion 20 (2011) 874–883. OH 44325-2102, , 2012.
[37] M. Siegel, P. Gunatilake, Remote enhanced visual inspection of aircraft by a mo- [63] A. Singh, Digital change detection techniques using remotely-sensed data, Inter-
bile robot, IEEE Workshop on Emerging Technologies, Intelligent Measurement national Journal of Remote Sensing 10 (6) (1989) 989–1002.
and Virtual Systems for Instrumentation and Measurement, ETIMVIS'98, St. [64] A. Kano, K. Doi, H. MacMahon, D.D. Hassell, M.L. Giger, Digital image subtraction of
Paul, MN, USA, May 1998. temporally sequential chest images for detection of interval change, 21 (1994) 453.
[38] M.R. Jahanshahi, S.F. Masri, Adaptive vision-based crack detection using 3D scene [65] G. Li, H. Xie, H. Ning, D. Citrin, J. Capala, R. Maass-Moreno, P. Guion, B. Arora, C.N.
reconstruction for condition assessment of structures, Automation in Construc- Coleman, K. Camphausen, R.W. Miller, Accuracy of 3D volumetric image registra-
tion 22 (2012) (2012) 567–576. tion based on CT, MR and PET/CT phantom experiments, Journal of Applied Clin-
[39] T. Zuhang, G. Jie, L. Yun, Y. Zhouping, A new image-based method for concrete ical Medical Physics 9 (4) (2008).
bridge bottom crack detection, Image Analysis and Signal Processing (2011), [66] O. Abudayyeh, M. Al Bataineh, I. Abdel-Qader, An imaging data model for concrete
http://dx.doi.org/10.1109/IASP.2011.6109108. bridge inspection, Advances in Engineering Software 35 (8–9) (2004) 473–480.
[40] T. Yamaguchi, S. Hashimoto, Practical image measurement of crack width for real [67] R.S. Adhikari, O. Moselhi, A. Bagchi, Automated prediction of condition state rating in
concrete structure, Electronics and Communication in Japan 92 (10) (2009). bridge inspection, ISG-ISARC Conference, Endhoven, Netherlands, Gerontechnology,
[41] R.S. Adhikari, O. Moselhi, A. Bagchi, Image-based retrieval of concrete crack prop- 11(2), 2012, p. 81, http://dx.doi.org/10.4017/gt.2012.11.02.153.00.
erties, ISG-ISARC Conference, Endhoven, Netherlands, Gerontechnology, 11(2), [68] F. Remondino, S. El-Hakim, Image-based 3d modelling: a review, The Photogram-
2012, p. 315. metric Record 21 (115) (September 2006) 269–291.
[42] C. Solomon, T. Breckon, A Book on Fundamentals of Digital Image Processing, a [69] S. McRobbie, R. Woodward, A. Wright, Visualisation and display of automated
Practical Approach with Examples in MatLab. this first edition published 2011 bridge inspection results, Transport Research Laboratory, UK NJDOT, 2009,
by John Wiley & Sons, Ltd, 2011. Bridge Inspection Work Zone, Setup Guide, 2010. (http://www.state.nj.us/
[43] M.D. Paul, B.H. Harry, S.F. Clive, R. Bjorn, N. Wolfgang, An operational application transportation/eng/structeval/pdf/InspectionWorkZoneSetup.pdf).
of automatic feature extraction: the measurement of crack in concrete structures, [70] Z. Zhu, Automated as-built modeling with spatial and visual data fusion, 12th In-
17 (99) (April, 2002) 453–464. ternational Conference on Construction Applications of Virtual Reality, Nov. 1–2,
[44] I. Laptev, H. Mayer, T. Linderberg, W. Eckstein, C. Steger, A. Maumgartner, Auto- Taipei, Taiwan, 2012.
matic extraction of roads from aerial images based on scale space and snakes, [71] M. Maksymowicz, P.J.S. Cruz, J. Bien, Load capacity of damaged RC slab spans of
Machine vision and Applications 12 (1) (2000) 23–31. railway bridges, Archives of Civil and Mechanical Engineering XI (4) (2011).
[45] C. Scheffy, N. Coetze, E. Diaz, Asphalt concrete fatigue crack monitoring and [72] K. Ramesh, Engineering Fracture Mechanics, Online Video Lectures, IIT Madras,
analysis using digital image analysis techniques, Accelerated Pavement Testing http://nptel.iitm.ac.in/courses/112106065/2.
1999 International Conference October 18–20, 1999, Reno, Nevada, October [73] Z. Zhu, S. German, I. Brilakis, Detection of large-scale concrete columns for auto-
18–20 1999. mated bridge inspection, Automation in Construction 19 (2010) 1047–1055.
[46] S.G. McRobbie, Automated inspection of highway structures 2008/09, Published [74] R.S. Adhikari, Z. Zhu, O. Moselhi, A. Bagchi, Automated Bridge Condition Assess-
Project Report PPR 412, Transportation Research Laboratory, UK, 2008/09. ment using Hybrid Sensing, ISARC 2013, 2013. (accepted on April 19, 2013).

Please cite this article as: R.S. Adhikari, et al., Image-based retrieval of concrete crack properties for bridge inspection, Automation in Construction
(2013), http://dx.doi.org/10.1016/j.autcon.2013.06.011