Crack Detection of Structures Using Deep Learning Framework: December 2020
Crack Detection of Structures Using Deep Learning Framework: December 2020
net/publication/346714622
CITATIONS READS
24 2,929
4 authors, including:
Ashutosh Trivedi
Delhi Technological University
77 PUBLICATIONS 710 CITATIONS
SEE PROFILE
All content following this page was uploaded by Avinash Kumar Jha on 13 February 2021.
Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 13,2021 at 05:17:56 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Third International Conference on Intelligent Sustainable Systems [ICISS 2020]
IEEE Xplore Part Number: CFP20M19-ART; ISBN: 978-1-7281-7089-3
Min-Max Gray Level Discrimination (M2GLD). This The dataset possesses images of concrete. The dataset
strategy's downside was that the clients need to fine-tune has two categories, i.e., positive and negative. The positive
boundaries changing proportion (RA) and edge boundary (τ). category contains cracked images, and the negative has non-
By utilizing this examination, it is discovered that RA = 2 and cracked images of concrete. The dataset contains 40,000
τ = 0.5 can convey acceptable crack identification on the images of concrete of pixel size 227*227 [18].
building surface. Be that as it may, for various kinds of
pictures, such as black-top or Asphalt, these two qualities 3.1.3 Asphalt Crack Dataset
should be balanced. Zheng et al. [6], to boost and extract the This dataset consists of 400 images of Asphalt cracks
data features, they have suggested FCN, RCNN, and RFCN and Non-Cracks images, that are captured using Nikon
frameworks. Although the accuracy is improved with the help camera, and can be used for training crack detection models
of RCNN and FCN but the framework takes more time to [13].
train. Fan et al. [7] proposed U Hierarchical Dilated Network
(U-HDN) to detect cracks. U Hierarchical Network uses the 3.2 Deep Learning Approach
architecture of encoder-decoder along with hierarchical
feature learning. However, U Hierarchical Network although 3.2.1 Modified LeNet-5
produces acceptable results, but due to the neural network's Following Table 1 depicts the architecture of the
complicated structure, the computational cost is high, and the modified version of LeNet-5 model when applied on
efficiency is low. Li et al. [8] concocted a technique for Concrete Crack Images for Classification Dataset. Input
distinguishing the surface breaks for stacked coal with the shape to the first layer of the model is of size (*, 227, 227, 3),
help of a vibration failure process, which is based on a here * represents the total number of images used while
vibration failure system and SVM. However, they did not use training the model.
Principal Component Analysis to analyze further which
feature of the crack plays a major role so that they can reduce
TABLE I. ARCHITECTURE OF MODIFIED LE-NET5
the training time. Chen et al. [9] detected cracks for the
submerged metallic plane from nuclear inspection videos Layer 𝒑𝒂𝒕𝒄𝒉 𝒔𝒊𝒛𝒆
with the help of a deep learning model called NB-CNN. Output Shape Param #
(Type) 𝒔𝒕𝒓𝒊𝒅𝒆
However, they did not use Principal Component Analysis to
further reduce their training time. Zhou et al. [10] analyzed Conv2D (5 × 5)/1 (*, 223, 223, 6) 456
three-dimensional (3D) laser image range data with an
automated crack detection technique, which uses frequency Average
domain filtering and contouring analysis. However, the (2 × 2)/2 (*, 111, 111, 6) 0
Pooling2D
above-proposed method fails in case of detecting shallow
cracks. Lin et al. [11], with the help of a sandstone specimen Conv2D (5 × 5)/1 (*, 107, 107, 16) 2416
under three-point bending, determined the cohesive zone's
characteristics and cohesionless crack's length. However, the Average
primary outcomes need additional examinations because the (2 × 2)/2 (*, 53, 53, 16) 0
Pooling2D
results of some numerical modeling get challenged. Zhang et
al. [12] compared the Handcrafts method's crack detection Flatten _ (*, 44944) 0
performance with the deep learning model and concluded that
the latter performed better. The dataset that they worked upon Dense _ (*, 120) 5393400
was small, which consisted of just 500 images.
Dense _ (*, 84) 10164
3. METHODOLOGY AND EXPERIMENTATION
Dense _ (*, 1) 85
3.1 Datasets
Total Parameters # 5,406,521 (5.4 M)
3.1.1 Automated Bridge Crack Detection Dataset
The original dataset comprises of 2068 Bridge Crack Trainable Parameters # 5,406,521 (5.4 M)
Images, collected by the Phantom 4 Pro's CMOS surface
array camera, with a resolution of 1024*1024 [14]. Non-Trainable Parameters # 0
3.1.2 Concrete Crack Images for Crack Classification 3.2.1.1 Convolutional Layer
Dataset
Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 13,2021 at 05:17:56 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Third International Conference on Intelligent Sustainable Systems [ICISS 2020]
IEEE Xplore Part Number: CFP20M19-ART; ISBN: 978-1-7281-7089-3
The convolutional layers are the primary building function and AdamOptimiser as the optimiser. The formula
blocks of a convolutional neural network structure. It is the of BinaryCrossEntropy loss function:
convolutional layers that learn the features that are suitable
for differentiating between a crack and non-crack image.
Initial convolutional layers learn the local features required
(3)
for this, whereas the deeper layers learn from the global
features required for differentiating cracks and non-cracks
[4]. The stride is the additional hyperparameter of the layer. where, CE is Binary Cross Entropy, C1 and C2 are are the
The number of receptive field columns and rows covering the two classes[0, 1](in our case, it is crack or non-crack), t1
input width and height is defined by stride. Fewer receptive and s1 are groundtruth and score for C1 class respectively
field applications and smaller output sizes are due to larger and t2(1-t1) and s2(1-s2) are groundtruth and score for C2
stride, which reduces computational cost at the price of loss class respectively.
of some input data features.
Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 13,2021 at 05:17:56 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Third International Conference on Intelligent Sustainable Systems [ICISS 2020]
IEEE Xplore Part Number: CFP20M19-ART; ISBN: 978-1-7281-7089-3
first eigenvector will give first max variance, and the second Principal Component Analysis was applied and at 10th epoch
eigenvector will give second max and so on. Eigenvalues will when Principal Component Analysis was not applied. To
represent how much variance will be explained by a certain overcome overfitting, the orientation of every image is
dimension. In case the eigenvalue of a certain direction is changed before passing it to the model, so that the model
very less, which means it has very less variance or never sees the same image again, for all the datasets that we
information hence it can be removed. have used. Figure 3,Figure 4, Figure 5 shows predictiction of
model on Concrete Crack Images for Classification Dataset,
3.4 Training & Testing of the Model Asphalt Crack Dataset and Automated Bridge Crack
We have applied modified LeNet-5 on the Automated Detection Dataset respectively, without using Principal
Bridge Crack Detection dataset, Concrete Crack Images Component Analysis. Whereas Figure 6, Figure 7, Figure 8
Classification dataset, and Asphalt Crack dataset. Figure 2 shows the prediction of the model on Concrete Crack Images
shows the Detection of Concrete Cracks Flowchart. The idea for Classification Dataset, Asphalt Crack Dataset and
of using modified LeNet-5 was to check the performance of Automated Bridge Crack Detection Dataset respectively, by
LeNet-5 by detecting the cracks on different types of using Principal Component Analysis.
materials. We wanted to make sure that modified LeNet-5
produces consistent results for all types of materials that we 3.5 Predictions on Images
have chosen to work upon. The modified LeNet -5 was
implemented using the Keras framework. 3.5.1 Predictions on images without using Principal
Component Analysis
Figure 3 shows the predictions of the model on
Concrete Crack Images for Classification dataset without
using Principal Component Analysis. Figure 4 shows the
predictions of the model on Asphalt Crack dataset without
using Principal Component Analysis. Figure 5 shows the
predictions of the model on Automated Bridge Crack
Detection dataset without using Principal Component
Analysis.
Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 13,2021 at 05:17:56 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Third International Conference on Intelligent Sustainable Systems [ICISS 2020]
IEEE Xplore Part Number: CFP20M19-ART; ISBN: 978-1-7281-7089-3
𝑇𝑃(𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠)
𝑅𝑒𝑐𝑎𝑙𝑙(𝑅) = (5)
𝑇𝑃(𝑡𝑟𝑢𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒𝑠) + 𝐹𝑁(𝑓𝑎𝑙𝑠𝑒 𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒𝑠)
4.1.3 Accuracy
It calculates how often the predictions are correct. The
formula for Accuracy is:
𝑇𝑃 + 𝑇𝑁
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦(𝐴) = (6)
𝑇𝑃 + 𝐹𝑃 + 𝑇𝑁 + 𝐹𝑁
Figure 12. False Positive Prediction
TP, TN, FP, and FN are True Positive, True Negative,
False Positive, and False Negative respectively. In our case,
Positive is when the image doesn’t have cracks and Negative
is when the image does have cracks. True Positive is when
the image doesn’t have cracks and the model predicted it as
non-crack. True Negative is when the image does have cracks
and the model also predicted it has cracks. False Positive is
when the image doesn’t have cracks but the model predicted
that it has cracks. False Negative is when the image does have
cracks but the model predicted that it has no-cracks. True
Positive Prediction, True Negative Prediction, False Positive
Prediction, False Negative Prediction is shown by Figure
Figure 13. False Negative Prediction
10,11,12 13 respectively.
4.2 Training Time for judging the performance of the model
Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 13,2021 at 05:17:56 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Third International Conference on Intelligent Sustainable Systems [ICISS 2020]
IEEE Xplore Part Number: CFP20M19-ART; ISBN: 978-1-7281-7089-3
6. ACKNOWLEDGEMENTS
The authors would like to thank the Department of Civil
Engineering, Delhi Technological University (DTU) and
Central Library, DTU for providing access to all the resources
used for our study. The financial support received under the
research project grant (F. No. DTU/IRD/619/2105) of DTU,
New Delhi is thankfully acknowledged
Figure 17. Training Time Curve
Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 13,2021 at 05:17:56 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Third International Conference on Intelligent Sustainable Systems [ICISS 2020]
IEEE Xplore Part Number: CFP20M19-ART; ISBN: 978-1-7281-7089-3
Figure 3. Predictions on Crack images for Classification dataset without using PCA
Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 13,2021 at 05:17:56 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Third International Conference on Intelligent Sustainable Systems [ICISS 2020]
IEEE Xplore Part Number: CFP20M19-ART; ISBN: 978-1-7281-7089-3
REFERENCES Data.mendeley.com<https://data.mendeley.com/data
sets/xnzhj3x8v4/2> (Sep. 27, 2020)
[1] Eisenbach, M., Stricker, R., Seichter, D., Amende,
[14] Xu, H., Su, X., Wang, Y., Cai, H., Cui, K., and Chen,
K., Debes, K., Sesselmann, M., Ebersbach, D.,
X. (2019). "Automatic Bridge Crack Detection Using
Stoeckert, U., and Gross, H. (2017) "How to get
a Convolutional Neural Network". Applied Sciences,
pavement distress detection ready for deep learning?
9(14), 2867
A systematic approach" 2017 International Joint
[15] Trivedi, A. (2015). "Computing in-situ strength of
Conference on Neural Networks (IJCNN)
rock masses based upon RQD and modified joint
[2] Yokoyama, S., and Matsumoto, T. (2017)
factor: Using pressure and damage sensitive
"Development of an Automatic Detector of Cracks in
constitutive relationship". Journal of Rock Mechanics
Concrete Using Machine Learning". Procedia
and Geotechnical Engineering, 7(5), 540-565
Engineering, 171, 1250-1255
[16] A. K. Jha, S. Adhikari, S. Thapa, A. Kumar, A.
[3] Cha, Y., Choi, W., and Büyüköztürk, O. (2017).
Kumar, and S. Mishra, “Evaluation of Factors
"Deep Learning-Based Crack Damage Detection
Affecting Compressive Strength of Concrete using
Using Convolutional Neural Networks" Computer-
Machine Learning” in 2020 Advanced Computing
Aided Civil and Infrastructure Engineering
and Communication Technologies for High
Engineering, 32(5), 361-378
Performance Applications (ACCTHPA), 2020:
[4] Pauly, L., Peel, H., Luo, S., Hogg, D., and Fuentes,
IEEE, pp. 70-74.
R. (2017) "Deeper Networks for Pavement Crack
[17] Tyagi, L., Butola, R., and Jha, A. (2020).
Detection" in 34th International Symposium on
"Mechanical and tribological properties of AA7075-
Automation and Robotics in Construction (ISARC)
T6 metal matrix composite reinforced with ceramic
[5] Hoang, N. (2018). "Detection of Surface Crack in
particles and aloevera ash via Friction stir
Building Structures Using Image Processing
processing". Materials Research Express, 7(6),
Technique with an Improved Otsu Method for Image
066526
Thresholding". Advances in Civil Engineering, 2018.
[18] Özgenel, Ç. (2020). "Concrete Crack Images for
[6] Zheng, M., Lei, Z., and Zhang, K. (2020) "Intelligent
Classification". Data.mendeley.com,
detection of building cracks based on deep learning"
<https://data.mendeley.com/datasets/5y9wdsg2zt/1>
Image and Vision Computing, 103, 103987
(Sep. 27, 2020)
[7] Fan, Z., Li, C., Chen, Y., Wei, J., Loprencipe, G.,
[19] A. Ghimire, S. Thapa, A. K. Jha, A. Kumar, A.
Chen, X., and Di Mascio, P. (2020) "Automatic Crack
Kumar, and S. Adhikari, “AI and IoT Solutions for
Detection on Road Pavements Using Encoder-
Tackling COVID-19 Pandemic”, in 2020
Decoder Architecture" Materials, 13(13), 2960
International Conference on Electronics,
[8] Li, C., and Ai, D. (2017). "Automatic crack detection
Communication and Aerospace Technology, 2020:
method for loaded coal in vibration failure process".
IEEE.
PLOS ONE, 12(10), e0185750
[20] A. Ghimire, S. Thapa, A. K. Jha, S. Adhikari, and A.
[9] Chen, F., and Jahanshahi, M. (2018). "NB-CNN:
Kumar, “Accelerating Business Growth with Big
Deep Learning-Based Crack Detection Using
Data and Artificial Intelligence”, in 2020 Fourth
Convolutional Neural Network and Naïve Bayes
International conference on I-SMAC (IoT in Social,
Data Fusion". IEEE Transactions on Industrial
Mobile, Analytics and Cloud) (I-SMAC), 2020:
Electronics, 65(5), 4392-4400
IEEE, pp. 441-448.
[10] Zhou, S., and Song, W. (2020). "Robust Image-Based
[21] S. Thapa, P. Singh, D. K. Jain, N. Bharill, A. Gupta,
Surface Crack Detection Using Range Data". Journal
and M. Prasad, “Data-Driven Approach based on
of Computing in Civil Engineering, 34(2), 04019054
Feature Selection Technique for Early Diagnosis of
[11] Lin, Q., Wan, B., Wang, S., Li, S., and Fakhimi, A.
Alzheimer's Disease”, in 2020 International Joint
(2019). "Visual detection of a cohesionless crack in
Conference on Neural Networks (IJCNN), 2020:
rock under three-point bending". Engineering
IEEE, pp. 1-8.
Fracture Mechanics, 211, 17-31
[22] S. Thapa, S. Adhikari, A. Ghimire, and A. Aditya,
[12] Zhang, L., Yang, F., Daniel Zhang, Y., and Zhu, Y.
“Feature Selection Based Twin-Support Vector
(2016). "Road crack detection using deep
Machine for the diagnosis of Parkinson’s Disease”, in
convolutional neural network". 2016 IEEE
2020 8th R10 Humanitarian Technology Conference
International Conference on Image Processing (ICIP)
(R10-HTC), 2020: IEEE
[13] A, J., G, T., M S, D., Nair, B., and D. S, H. (2020).
"Asphalt Crack Dataset".
Authorized licensed use limited to: DELHI TECHNICAL UNIV. Downloaded on February 13,2021 at 05:17:56 UTC from IEEE Xplore. Restrictions apply.
View publication stats