See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/322840468

A Method of Crack Detection Based on Convolutional Neural Networks

Conference Paper · September 2017

DOI: 10.12783/shm2017/13959

CITATIONS READS
11 785

2 authors, including:

Shengyuan Li
China University of Mining and Technology
18 PUBLICATIONS   540 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Computer vision-based structure health monitoring View project

All content following this page was uploaded by Shengyuan Li on 18 February 2019.

The user has requested enhancement of the downloaded file.

 COVER SHEET

Title: A Method of Crack Detection Based on Convolutional Neural Networks

Authors: Xuefeng Zhao

Shengyuan Li
ABSTRACT

Crack is one of important features in gradually degeneration process for structures,

and the cracks of structure can to some extent reflect the health status of the structure.
Moreover, convolutional neural networks (CNNs) provide an ideal route to detect
cracks utilizing the explosion of Internet and the improvement of computer hardware.
In this paper, a method for crack detection based on CNNs is proposed. Firstly, to
provide datasets for CNN model training, a large number of crack images ware
collected, cut, and labeled. Secondly, a CNN model for crack detection was
established through the fine-tuning AlexNet model. Thirdly, the CNN model was
trained by the crack images datasets. The trained CNN model is used as a classifier
and the crack detection was implemented using the CNN classifier and a sliding
window technique. Finally, a crowdsourcing to collect crack images and detect crack
is proposed. The objectives of this work are to set up a crack images database for more
researchers and obtain a more and more accurate and trained classifier for crack
detection via CNNs, and mobilizing the public to collect crack images and detect
crack using the trained CNN classifier through the crowdsourcing.

INTRODUCTION

The crack is one of the most common damages on structures. Traditionally, the
primary method to detect cracks is visual inspection. During crack detection, a trained
inspector is employed to evaluate the condition of the structure according to location
and width of the crack. This is an effective method for many situations. However, this
is a subjective evaluation of the current condition based on some guidelines and the
inspector’s experience. As a result, the evaluation will be widely varied from inspector
to inspector [1]. Based on this, image processing techniques is widely used to detect
superficial damages, such as cracks [2, 3], corrosion and so on [4]. The major research
production of crack detection utilizing image processing techniques can be divided
into two levels, i.e., identifying cracks from image background and measuring width

Xuefeng zhao, School of Civil Engineering, Dalian University of Technology, No. 2, Ling Gong
Road, Ganjingzi District, Dalian, China
Shengyuan Li, School of Civil Engineering, Dalian University of Technology, No. 2, Ling Gong
Road, Ganjingzi District, Dalian, China
of the crack after crack identifying. This method depends on just right image
pretreatment technique [5], and the edge detection also plays an important role in this
process [6, 7]. For further study, unmanned aerial vehicles also are used to collect the
crack images [8]. Measuring the width after crack identifying need compute a ratio
between pixel distance and actual distance [9]. However, all of the methods based on
image processing techniques detect crack in particular situations using particular skills,
but the situations in real world varies extensively. That is to say, the adaptability of
methods based on image processing techniques is not very strong.
To solve this problem, a possible route to detect cracks that has more real-world
situation adaptability is using machine learning algorithms [10, 11]. The machine
learning algorithms take the advantage of big data, so they are not sensitive to
abnormal data. One of them, convolutional neural networks (CNNs), is especially
excellent in classification of images [12]. The CNNs produce fewer parameters than
standard neural networks because of local connection. Besides, the application of
CNNs is restricted by large amount of labeled images and computer property in the
past, but those two problems were solved by labeled-databases (MNIST DATABASE,
CIFIA-10 and CIFAR-100 dataset, ImageNet) and graphic processing unit (GPU).
In this paper, a method of crack detection based on CNNs is proposed. The one
objective of this study is to set up a database of cracks for more researchers. The
further study about crack detection can be implemented using the database. Another
objective is to build a robust classifier for with crack images and no crack images that
will be almost not affected by shadow and changing of lighting, and so on. Section 2
introduces a framework of the proposed method of crack detection based on CNNs.
Section 3 presents a architecture of the CNN used in this paper. In section 4, the
realization of crack detection using the CNN is described in detail. Section 5 proposes
the crowdsourcing that the public can be mobilized to collect crack images and detect
crack using the trained classifier. Section 6 is the conclusion of this paper.

Collecting crack images

Building datasets

Training set Validation set

Training a CNN

CNN classifier Sliding window technique

Crack detection

Figure 1. The framework of crack detection based on convolutional neural networks

FRAMEWORK OF THE PROPOSED METHOD

Figure 1 demonstrates a framework of crack detection based on CNNs. To train a

CNN classifier, large amount of crack images are collected firstly. After that, the
images are cropped into small images including crack and no crack, and label of each
image is set to build training and validation sets. Then the prepared training and
validation sets are used to train the CNN to generate a classifier for crack and no crack
images. Finally, the crack is detected using the CNN classifier and a sliding window
technique.

ARCHITECTURE OF THE CNN

In this paper, the Caffe [13], a deep learning framework, is used to finish the study.
The AlexNet model was employed [14], and the fine turning strategy was utilized [15].
As shown in the figure 2, because the classes in our study are crack and no crack,
the output classes are set to 2. The pixel resolutions of input images are 256×256×3,
where each dimension expresses height, width, and channel of the images respectively.
The input images pass through the architecture, and a series of operations are
implemented, including convolution, pooling, rectified linear unit (ReLU),
normalization, and fully connected layers. Finally, the softmax predicts whether each
input image is a crack or no crack image.
The dimensions of data will change after every operation when the CNN are
trained. Table I shows detailed dimensions of each convolution and polling, and Table
II demonstrates the dimensions of data after suffering from each operation in Figure 2.

Image
conv1 relu1 pool1 norm1 data1
input

conv2 relu2 pool2 norm2 data2

conv3 relu3 data3

conv4 relu4 data4

conv5 relu5 pool5 data5

fc6 relu6 drop6 data6

fc7 relu7 drop7 data7

with crack
fc8 data8 softmax
no crack

Figure 2. The architecture (conv#: convolution; relu#: ReLU layer; pool#: pooling; norm#:
normalization; data#: data; fc#: fully connected layer; drop#: dropout layer)
 TABLE I. DIMENSIONS OF EACH OPERATION
Operation No. Pad Kernel size Stride
conv1 96 0 11 4
pool1 - 0 3 2
conv2 256 2 5 1
pool2 - 0 3 2
conv3 384 1 3 1
conv4 384 1 3 1
conv5 256 1 3 1
pool5 - 0 3 2

TABLE II. DIMENSIONS OF DATA

Data
input data1 data2 data3 data4 data5 data6 data7 data8
Dimensions
Height 227 27 13 13 13 6 1 1 1
Width 227 27 13 13 13 6 1 1 1
Depth 3 96 256 384 384 256 4096 4096 2

TRAINING A CNN CLASSIFIER FOR CRACK DETECTION

The method of crack detection in this paper mainly depends on a CNN classifier.
In this section, the training of the classifier for crack detection is described in detail.
All of study in this paper is implemented on a workstation with a GPU (CPU: Intel
Xeon CPU E5-2630 v4 @2.20GHz, RAM: 32GB, GPU: GeForce GTX 1080 Ti).

Building datasets

The dimensions of row images in this paper are 4160×3120. Those row images are
cropped into the images with 256×256 pixel resolutions to build the training set and
validation set. As shown in Figure 3, the small images are classified into two classes:
with crack and no crack, and they are labeled according to the classification result.
Because the number of no crack images is far more than the images with crack, some
no crack images are deleted randomly to make the ratio of the number of with crack
and no crack images be 1:1. After the above processing steps, the number of images in
training set and validation set is 21000 and 4200 respectively. Besides, the mean of
images in training and validation sets are computed respectively, and each pixel in the
images is subtracted from the corresponding mean.

(a) With crack (b) No crack

Figure 3. Examples of images in datasets
 Figure 4. Training result

Training the CNN

The proposed network is trained using a stochastic gradient descent algorithm.

The base learning rate and learning rate policy are assigned by 0.001 and “step”, and
weight decay and momentum parameters are set to 0.0001 and 0.9 respectively. The
dropout ratio at the dropout layer is 0.5 and the maximum number of iterations is
10000. The training result are shown in Figure 4, it illustrates test accuracy rapidly
increases and finally plateaus at 98.6667%. The trained CNN model of the 10000st
iteration is used as a classifier for crack detection.

Crack detection using the trained CNN classifier

To detect crack using the trained CNN classifier, extensive search is conducted.
As shown in the Figure 5, a sliding window technique where the dimensions of
window are 256×256 pixels and all of the horizontal and vertical strides are 128 pixels
is designed. As shown in Figure 6, a row image of 4160×3120 pixel resolutions is
scanned by a sliding window of 256×256 pixels. The small image at every window is
predicted by the trained CNN classifier, and no crack images are removed. The
scanning result shows the trained convolutional neural networks classifier is robust to
detect crack.

128 256
256 128

Figure 5. Sliding window technique

 (a) Row image (b) Scanning result
Figure 6. Crack detection

Storing Training a
images CNN classifier

2G,3G,4G/wifi

Taking, cutting,
classifying, and
uploading images
Crack
detection

Figure 7. A crowdsourcing of images collection and crack detection

A CROWDSOURCING OF CRACK IMAGES COLLECTION AND CRACK

DETECTION

The popularity of smart phones provides an opportunity for big data collection.
The collection of crack images can no longer rely solely on professional inspector. As
shown in Figure 7, the public can be mobilized to collect, cut, classify, and upload
crack images via smartphone. With the expansion of the database of crack images, a
classifier with growing accuracy will be trained utilizing the same process in this
paper, and the public can detect crack using the trained classifier via smartphone.

CONCLUSION

In this paper, a CNN classifier for crack detection was built. The row images with
4160×3120 pixel resolutions were cropped into the images with 256×256 pixel
resolutions to build the training set and validation set. The trained CNN model came
from the fine-tuned AlexNet model. Test accuracy of the trained CNN was 98.6667%.
A sliding window technique where the dimensions of window ware 256×256 pixels
was designed to detect crack in an image with 4160×3120 pixel resolutions, and the
scanning result illustrated that the trained classier was robust to detect crack.
REFERENCE
1. Phares, B. M., Rolander, D. D., Graybeal, B. A., and Washer, G. A. 2001. “Reliability of visual
bridge inspection,” Public Roads, 64(5).
2. Yamaguchi, T., and Hashimoto, S. 2010. “Fast crack detection method for large-size concrete
surface images using percolation-based image processing,” Machine Vision and Applications, 21(5):
797-809.
3. Yamaguchi, T., Nakamura, S., Saegusa, R., and Hashimoto, S. 2008. “Image‐Based Crack
Detection for Real Concrete Surfaces,” IEEJ Transactions on Electrical and Electronic
Engineering, 3(1): 128-135.
4. Codaro, E. N., Nakazato, R. Z., Horovistiz, A. L., Ribeiro, L. M. F., Ribeiro, R. B., and Hein, L. D.
O. 2002. “An image processing method for morphology characterization and pitting corrosion
evaluation,” Materials Science and Engineering: A, 334(1): 298-306.
5. Fujita, Y., Mitani, Y., and Hamamoto, Y. 2006. “A method for crack detection on a concrete
structure,” In Pattern Recognition, 2006. ICPR 2006. 18th International Conference, pp. 901-904.
IEEE.
6. Abdel-Qader, I., Abudayyeh, O., and Kelly, M. E. 2003. “Analysis of edge-detection techniques for
crack identification in bridges,” Journal of Computing in Civil Engineering, 17(4): 255-263.
7. Nishikawa, T., Yoshida, J., Sugiyama, T., and Fujino, Y. 2012. “Concrete crack detection by
multiple sequential image filtering,” Computer‐Aided Civil and Infrastructure Engineering, 27(1):
29-47.
8. Kim, J. W., Kim, S. B., Park, J. C., and Nam, J. W. 2015. “Development of Crack Detection System
with Unmanned Aerial Vehicles and Digital Image Processing,” Advances in structural engineering
and mechanics (ASEM15).
9. Sohn, H. G., Lim, Y. M., Yun, K. H., and Kim, G. H. 2005. “Monitoring crack changes in concrete
structures,” Computer‐Aided Civil and Infrastructure Engineering, 20(1): 52-61.
10. Chen, F. C., Jahanshahi, M. R., Wu, R. T., and Joffe, C. 2017. “A texture‐Based Video Processing
Methodology Using Bayesian Data Fusion for Autonomous Crack Detection on Metallic Surfaces,”
Computer‐Aided Civil and Infrastructure Engineering, 32(4): 271-287.
11. Kaseko, M. S., Lo, Z. P., and Ritchie, S. G. 1994. “Comparison of traditional and neural classifiers
for pavement-crack detection,” Journal of transportation engineering, 120(4): 552-569.
12. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. 1998. “Gradient-based learning applied to
document recognition,” Proceedings of the IEEE, 86(11): 2278-2324.
13. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama S., and Darrell,
T. 2014. “Caffe: Convolutional architecture for fast feature embedding,” In Proceedings of the 22nd
ACM international conference on Multimedia, pp. 675-678. ACM.
14. Krizhevsky, A., Sutskever, I., and Hinton, G. E. 2012. “Imagenet classification with deep
convolutional neural networks,” In Advances in neural information processing systems, pp. 1097-
1105.
15. Reyes, A. K., Caicedo, J. C., and Camargo, J. E. 2015. “Fine-tuning Deep Convolutional Networks
for Plant Recognition,” In CLEF (Working Notes).
View publication stats