Applied Acoustics 168 (2020) 107402

Contents lists available at ScienceDirect

Applied Acoustics
journal homepage: www.elsevier.com/locate/apacoust

Developing deep neural network for damage detection of beam-like

structures using dynamic response based on FE model and real healthy
state
Zohreh Mousavi a, Mir Mohammad Ettefagh a,⇑, Morteza H. Sadeghi a, Seyed Naser Razavi b
a
Faculty of Mechanical Engineering, University of Tabriz, Tabriz 5166616471, Iran
b
Faculty of Electrical and Computer Engineering, University of Tabriz, 5166616471 Tabriz, Iran

a r t i c l e i n f o a b s t r a c t

Article history: Fundamentally, Structural Health Monitoring (SHM) of mechanical systems is essential to avoid their
Received 4 December 2019 catastrophic failure. The first key contribution of this paper is presenting a new method for damage
Received in revised form 24 March 2020 detection of mechanical systems in presence of the uncertainties such as modeling errors, measurement
Accepted 23 April 2020
errors, varying loading conditions and environmental noises based on Finite Element (FE) model and real
healthy state. On the other hand, deep learning has been widely used in image and signal analyses with
great success. According to this enhancement, the second key contribution of this paper is designing a
Keywords:
developed Deep Convolutional Neural Network (DCNN) with training interference and customized archi-
Structural Health Monitoring
Deep Convolutional Neural Network
tecture to learn the features. In industrial environments, most structures are exposed to varying environ-
Feature Learning mental conditions and it is difficult to collect data containing real damages, and generally, only the data
Dynamic Response of a real healthy system is available; therefore, it is necessary to have an effective method for damage
Model Updating detection of real systems based on the artificial damages and real healthy data. From this standpoint,
Beam-Like Structures the third key contribution of this paper is training process of the proposed DCNN using raw frequency
data of the FE model and real healthy state, which is then tested using the raw frequency data of the real
system. The proposed DCNN can directly learn the features from raw frequency data of the FE model and
real healthy state and discover the damage-sensitive features in order to damage detection of a real sys-
tem. In this method, only dynamic responses of real healthy system are used to updating the FE model
and minimizing the errors. The efficacy of the proposed method is validated using the experimental beam
structure. Time data and several manual features from time and frequency data as well as two intelligent
methods are used as comparisons. The results show that the proposed method can learn the features
from raw frequency data and achieve higher accuracy than other comparative methods.
Ó 2020 Elsevier Ltd. All rights reserved.

1. Introduction structural decay. However, there is no optimal individual method

for all the problems.
In the industry, mechanical systems and structures are widely Feature extraction plays a key role in damage detection of
used and considered as essential equipment, which require Struc- mechanical systems. However, there are several problems in the
tural Health Monitoring (SHM) for monitoring and evaluating traditional methods of feature extraction, such as: requirements
structural health. Damage detection of structures in a timely man- of prior knowledge; sensitivity to changes of mechanical systems
ner is necessary, but the problem is that performing the mainte- and limitations of extracting new features. It is necessary to inves-
nance tests in such environmental conditions are costly and even tigate an automated method of feature extraction that can learn
sometimes impossible. Therefore, it is clear that development of features from raw data [1]. Machine learning is addressed as a tool
robust and appropriate methods for the identification of main to select the features in order to judge on a condition of the
damages of structure is very efficient in preventing of disastrous mechanical systems. Artificial Neural Network (ANN) is a set of
algorithms used in machine learning for modeling the data. The
use of ANN for damage detection provides novel methods for
SHM; the ANNs are divided into two general categories: Shallow
⇑ Corresponding author. Neural Networks (SNN) and Deep Neural Networks (DNN). DNNs
E-mail address: [email protected] (M.M. Ettefagh). have been widely used in image and signal analyses with great suc-

https://doi.org/10.1016/j.apacoust.2020.107402
0003-682X/Ó 2020 Elsevier Ltd. All rights reserved.
2 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402

cess. 2D-DNNs are used for image-based data [2–4], and 1D-DNNs cally extract features using deep learning. A convolutional deep
are used for signal-based data. The use of 1D-DNN based on vibra- network was designed to learn features and damage detection of
tion signals (in time domain or frequency domain) for damage beam structure under a random excitation. FE numerical investiga-
detection of mechanical systems and structures has previously tions on beam structure were conducted to demonstrate the effi-
been used [1,5–10]. The key benefit of the DNNs is their ability ciency and high accuracy of the mentioned algorithm. To obtain
to extract damage-sensitive features from raw data. In fact, DNNs a physical insight into how the network works, visualization of
provides the abilities of feature extraction, feature selection and the features learned by hidden layers was investigated. Pathiragea
classification, and subsequently forms an end to end machine et al. [17] proposed an autoencoder neural network for structural
learning system. DNNs have more powerful feature learning capa- damage detection. Two main components were defined in the
bilities than the SNNs. As a result, they have better performance mentioned algorithm. The first was the dimensionality reduction
than the shallow architecture. Alongside these benefits, deep of the original input vector while preserving the essential informa-
learning algorithms require relatively big training data, which is tion. The second component was to perform the learning relation-
difficult and time-consuming to collect. In addition, DNNs often ship between the compressed features and the structure’s stiffness
usually require complicated hyper-parameters as well as a variety reduction parameters. To train this network, a pre-training scheme
of architecture choices that increase the complexity of constructing was performed, and fine-tuning was carried out to optimize the
an effective model [1]. Various deep learning architectures, such as entire network. In order to investigate the accuracy and effective-
Deep Convolution Neural Networks (DCNN), Deep Belief Networks ness of the algorithm mentioned above, numerical and experimen-
(DBN) and Deep Auto-Encoders (DAE) have been used to reduce tal analyses on steel frame structure were carried out separately.
dimensionality of input data and extract the features from them Vibration characteristics, such as natural frequencies and mode
[11]. A comprehensive overview of the DNNs and their applications shapes, are used as input, and structural damages are considered
for machine health monitoring has been published in Refs. [12,13]. as output vector. The algorithm achieved high accuracy in terms
Mechanical system data sets can be classified in two categories: of both the location and severity of the damages. When the simu-
artificial data sets and real data sets (which are based on data- lated data is used instead of the real data, one confronts evident
driven algorithms). Artificial data sets can be obtained easier differences between their dynamic characteristics. There is no
because they can be manually attained using machining instru- doubt that data generated by simulated model and real system
ments [10] or simulated models. Data-driven algorithms have share some common characteristics. So, it is necessary to update
effectively been successful in detecting mechanical system dam- the simulated model based on real healthy data and to extract
ages. Most of the data-driven techniques are based on algorithms common features between the simulated and real data. However,
of machine learning. Jing et al. [1] proposed a method based on fea- these common features cannot be well extracted by conventional
ture learning for damage detection of gearbox. In this method a feature extraction methods [10].
convolution neural network was developed to learn features from Mojtahedi et al. [18] investigated a robust SHM technique for
the frequency data, and checking the learning performance of dif- offshore jacket platform using model updating and fuzzy logic sys-
ferent features from time data, frequency data and combined tem. In this article, an experimental study was carried out on a lab-
time-frequency data were performed. The results showed that this oratory model of a jacket platform with the aim of creating a
approach is capable of learning the features of frequency data and baseline FE model for long-term SHM of this type of structures.
achieving higher detection accuracy than other comparative meth- The FE model was updated according to the experimental results
ods. Abdeljaber et al. [14] presented a new algorithm to damage to minimize the differences between the numerical and experi-
detection using the 1D Convolutional Neural Network. The men- mental characteristics which arise due to the different uncertain-
tioned algorithm was experimentally verified by monitoring the ties in the FE model and measured vibration data. The updated
main steel frame of the QU grandstand simulator by 30 accelerom- FE model was used to predict the changes in the dynamic charac-
eters, which were mounted on the 30 joints of main girders. The teristics in a variety of damage scenarios that were considered by
results showed a high level of performance in structural damage decreasing the stiffness at the components of the model, and the
detection processes. Undoubtedly, great attention should be paid effectiveness of the method was validated. Teng et al. [19], pre-
to the adaptability of damage detection algorithms in the industrial sented a CNN with the aim of extracting the damage features of
environments, which can be assessed in this aspect: when the a steel frame structure. A convolution operation was used to
working load of an actual system is changed, the trained algorithm extract the features of the modal parameters, and a classification
can still achieve high accuracy, if it is tested on samples from algorithm was used to assess the damage state of the structure.
another working load. Zhang et al. [9] presented a new approach The FE technique was used to acquire the first-order modal strain
based on deep learning to perform damage diagnosis under noisy energy for various damage states, which was used as training sam-
environments and various working loads. They presented an end- ples. Then, the experimental verification was carried out on a lab-
to-end method which took raw temporal signals as inputs. The scale model. The results showed that CNN was effective in detect-
mention method did not rely on any algorithms for domain adapta- ing intact, single damage, and multiple damage structure with high
tion or required target domain information. This method could accuracy. Ding et al. [20], introduced an approach for structural
achieve high accuracy when working load is changed. In industrial damage detection with uncertain FE modeling errors and measure-
environments, it is difficult and even impossible to collect data con- ment noises using a clustering-based tree seed algorithm (C-TSA).
taining real damages, and generally only data on a healthy state is The objective function was developed based on the modal data.
available, it is not surprising that data-driven methods are rapidly Numerical study on benchmark functions and the 61-bar truss
degrading in industrial environments. One feasible way to address structure as well as the experimental verification on a laboratory
the problem is using the data generated from artificial damages structure model were carried out. A modal hammer with a rubber
instead of real ones [10]. Consequently, using of the simulated tip was used to generate the impact excitation in numerical and
model approach and real healthy system can be a useful solution. experimental models. The dominant natural frequencies and mode
The general stages of a dynamic modeling approach are the shapes were used as the input. The results of the numerical and
selection of a mathematical model for a physical problem, develop- experimental investigations were compared with the results of
ing the model and providing its solution. Turner et al. [15] used the several recent evolutionary algorithms, which showed that the
Finite Element (FE) method to analyze the aircraft’s structure in above-mentioned method was more competitive and robust in
1956. Lin et al. [16] presented an approach in order to automati- the identification of structural damage.
 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402 3

Review of damage detection studies of mechanical systems noulli beam structure (FE Beam Model) is developed using the
based on vibration analysis shows that many studies have been per- Matlab software package. The term ‘‘initial” implies that the FE
formed in this regard so far, and in most studies, traditional meth- model could be inaccurate due to various uncertainties; therefore,
ods have typically been used to extract damage-sensitive features. the initial model is the basis for the model updating purpose. Fig. 1
Also, these studies are based on the using of experimental data or shows a bending element that x-y plane is the principle bending
mathematical model data individually; but using of experimental plate [25]. According to Fig. 1, the general displacement m in the
data and mathematical model data at the same time has been less y-direction is described as follows:
studied, which in these cases the model updating is mostly based
on real healthy data and real damage data. On this basis, a new uðtÞ ¼ v ð1Þ
approach is presented in this paper for damage detection of real
mechanical systems in presence of uncertainties such as modeling Force by (force unit length) in the y-direction is also described
errors, measurement errors, varying loading conditions and envi- as follows:
ronmental noises based on FE model and real healthy state. A devel-
oped Deep Convolutional Neural Network (DCNN) with training
bðtÞ ¼ by ð2Þ
interference and customized architecture is designed to learn the There are two nodal displacements q1 and q2 in point 1, q1 being
features. In industrial environments, most structures are exposed translational motion in y-direction and q2 being rotation motion in
to varying environmental conditions and it is difficult to collect z-direction. Similarly, there are two nodal displacements q3 and q4
the data containing real damages, and generally, only the data of in point 2, so the displacement vector is described as follows:
a real healthy system is available; therefore, in order to train the
proposed DCNN, raw frequency data (not the dominant natural fre- qðtÞ ¼ fq1 ; q2 ; q3 ; q4 g ¼ fv 1 ; hz1 ; v 2 ; hz2 g ð3Þ
quencies) of the FE model under simple working load conditions where v 1 and v 2 shows displacement in the y-direction and hz1 and
and real healthy state are used. Then, raw frequency data of the dif- hz2 shows slope in the z-direction in these points. Similarly, node
ferent states of real model under complex working load conditions forces are described as follows:
(for more realistic assumptions) are used to test the proposed

DCNN. The proposed DCNN can directly learn the features from pðtÞ ¼ fp1 ; p2 ; p3 ; p4 g ¼ py1 ; Mz1 ; py2 ; Mz2 ð4Þ
raw frequency data of the FE model and real healthy state and dis- where py1 and py2 shows force in the y-direction and Mz1 and Mz2
cover the damage-sensitive features in order to damage detection
shows torque in the z-direction in these points.
of the real system. In this method, data of real healthy system is
The matrix of the shape function for a bending element is con-
used for updating the parameters of the FE model and minimizing
sidered as follows [25]:
the errors. Some parts of the signals which are not related to the
nature of the system are removed using the Complete Ensemble 8 9
8 9 > 2 x33  3 x22 þ 1 >
Empirical Mode Decomposition (CEEMD) method. The raw fre- > f1 > > > L L >
>
>
> > > >
> x3 >
>
quency data is extracted from vibration signals using Frequency < f 2 = < L2  2 L þ x =
x2

Domain Decomposition (FDD) method. Actually, the proposed H¼ ¼ ð5Þ

>
>f3 > > > > 2 xL3 þ 3 xL2 >
3 2
>
method does not require any information from the damage states >
: > ; > > >
>
f4 >
: 3 2
>
;
2  L
of the real system to train DCNN. In the proposed DCNN, to suppress x x
L
high frequency noise, the wide kernel is used in the first convolu-
tion layer. Small kernels are used in the next layers for nonlinear Four shape functions are visible in Fig. 1. Shape functions repre-
multi-layer mapping. Also, the big step size is used in the first con- sent the variations v along the length of the element, L, due to
volutional layer to increase the speed of training of the proposed nodal displacements q1 to q4 . Shape functions are known as the
DCNN. The proposed method is validated using an experimental Hermitian Cubic Interpolation shape function. Using the Hermitian
beam structure. Different damage states are created by adding the Cubic Interpolation shape function, the mass and stiffness matrices
lumped masses with varying severities to the different elements of the beam elements can be extracted as follows [25]:
of structure. Time data and several manual features from time 2 3
and frequency data as well as two common intelligent methods, 156 22L 54  13L
namely Back-Propagation Neural Network (BPNN) [5,7,21,22] and 6
qAL 6 22L 4L2 13L 2 7
 3L 7
Me ¼ 6 7 ð6Þ
Support Vector Machine (SVM) [23,24], are used as comparisons. 420 4 54 13L 156  22L 5
The rest of the paper is organized as follows: In Section 2, the 13L  3L2  22L 4L2
numerical model and experimental setup of a beam structure,
and foundations CEEMD, FDD and DCNN methods are explained; 2 3
6 3L 6 3L
2EI 6 L2 7
the details of the proposed method are explained in Section 3; In 2L2  3L
6 3L 7
Section 4, the proposed method for damage detection of the exper- Ke ¼ 6 7 ð7Þ
imental beam structure is applied; and conclusions are outlined in L3 4 6  3L 6  3L 5
Section 5. 3L L2  3L 2L2
where I, q and E represents the moment of inertia (second moment
2. Materials and methods of area A), density and Young’s modulus. In systems with high
degrees of freedom, the damping matrix is typically defined as
In this section, at first the FE and experimental models of the the sum of the coefficients of mass and stiffness matrices as follows
beam structure is explained. Then, the mathematical backgrounds [26]:
of CEEMD, FDD and DCNN methods are expressed.  
½Cb  ¼ a ½Mb  þ b ½Kb  ð8Þ

2.1. Numerical model of beam structure (FE beam model) This type of damping is called proportional damping. Mb , Kb and
Cb displays the mass, stiffness and damping matrices of the whole
Considering the small deformation and linear behavior of the structure, respectively. For a system with high degrees of freedom,
 
system, an initial FE model of the simply supported Euler–Ber- values of a and b can be presented as follows [26]:
4 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402

Fig. 1. Bending element [25].

 2n1 x 1  2n2 x 2 The FE model of the simply supported Euler-Bernoulli beam struc-
b¼ ð9Þ
x 21  x 22 ture is showed in Fig. 2.
 
Also, the value of a is also obtained by inserting b in the follow- 2.2. Experimental setup of beam structure (experimental beam model)
ing equation [26]:
 
A model of a beam structure in laboratory scale (Experimental
2n1 x 1 ¼ a þ b x 21 ð10Þ
Beam Model) is used in this study for the experimental aspects
where n1 is the first mode damping ratio, x 1 is the first natural fre- which has been designed and installed in the Vibration and Modal
quency, n2 the second mode damping ratio and x 2 is the second Analysis Research Lab at the University of Tabriz. Experimental
natural frequency of the system. The vibration equation of the beam equipment included: a Pulse Data acquisition (3560C, B&K), two
can be written as follows [27]: shakers (4809, B&K), two accelerometers (4500, B&K), two force
transducers (208C03, PCB), a Windows-based laptop, two power
€ þ Cb Z_ þ Kb Z ¼ FðtÞ
Mb Z ð11Þ amplifiers (2706, B&K), and signal transfer cables (see Fig. 3).
where Z, Z_ and Z€ are the displacement, velocity and acceleration Two shakers are connected to the structure at the nodes No. 5
vectors of the beam structure. The ode45 method is used to solve and No. 8 (which shall be referred to as ‘‘shaker I and shaker II”
in this paper) via two flexible stingers with high axial stiffness,
Eq. (11). Equations are formulated as Y_ ¼ TðY; t Þ for the ode45
therefore the structure is not subjected to any moment of excita-
method. Therefore, Eq. (11) should be rewritten to this form. It is
tion or rotational loading [28]. The uncorrelated force signals used
assumed for this purpose:
for random excitation are white Gaussian noise in desired fre-
Z ¼ Y1 & Z_ ¼ Y2 & Y_ 1 ¼ Y2 ð12Þ quency range from 0 to 1600 Hz. It contains all frequencies within
the desired frequency range. Two force transducers are used to
So, Eq. (11) is rewritten as follows: recording the excitation forces. Two accelerometers I and II are
Mb Y_ 2 þ Cb Y2 þ Kb Y1 ¼ FðtÞ ð13Þ mounted along the beam at the nodes No. 6 and No. 7 for extract-
ing the dynamic responses of the structure (see Fig. 4). The force
The matrix form of Eq. (11) is written as follows: transducers and the accelerometers used in the equipment have
 ( _ )      small masses, which change just slightly the mass of the structure.
In 0 Y1 0  In Y1 0
þ ¼ ð14Þ The apparent mass of each transducer is only about 4gr, which is
0 Mb Y_ 2 Kb Cb Y2 FðtÞ less than 1% of the mass of the structure.
Fig. 5 shows different parts of the experimental setup of the
Finally, Eq. (11) is written as follows:
( ) beam structure. The accelerometers and the force transducers are
      
Y_ 1 In 0 0  In Y1 0 connected to the Pulse Data acquisition system with the signal
¼ inv  þ ð15Þ
Y_ 2 0 Mb Kb Cb Y2 FðtÞ transfer cables. Vibration signals for 20 s are recorded using the
Pulse Labshop software with a sampling frequency rate of
where In represents the identity matrix. Eq. (15) is a solvable form 8192 Hz, and then these signals are imported into the MATLAB
for the ode45 method. The dynamic responses of the beam struc- software for further processes. In this paper, first tests are per-
ture are extracted by placing Mb , Kb and Cb in Eq. (15). In this paper, formed on the healthy structure. Then the tests are repeated in
an Aluminum beam structure with a rectangular cross section area the same way for the different damage states of the structure by
(0.02047 m width, 0.01045 m thickness, and 0.82 m length) is used. adding the lumped masses with varying severities to the different
The model dimensions are created exactly with respect to the elements of the structure, which is explained in the following sec-
dimensions of the experimental model (see Fig. 4). The initial tions. It should be noted that the weight of the masses is consid-
Young’s modulus, poisson’s ratio and density are 70 GPa, 0.33 and ered as the damage severities. All above-mentioned processes are
2710 kg/m3, respectively. The beam structure is divided into 11 ele- carried out by the FE model, and the extracted data are analyzed
ments (12 nodes from No. 0 to No. 11) with the length of 0.074 m. similar to the experimental approaches. The only difference
 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402 5

Fig. 2. The FE model of the simply supported Euler-Bernoulli beam structure.

Fig. 3. Experimental equipment: (a) Pulse Data acquisition (3560C, B&K); (b) shaker (4809, B&K); (c) accelerometer (4500, B&K); (d) force transducer (208C03, PCB); (e)
windows-based laptop; (f) power amplifier (2706, B&K); (g) signal transfer cable.

Fig. 4. Location of accelerometers and shakers connected to the structure.

2.3. Complete ensemble empirical mode decomposition (CEEMD)

CEEMD [29] is a method of signal processing that breaks down

data into several Intrinsic Mode Functions (IMFs) through a process
known as the sift process. The IMFs must satisfy two conditions:
(1) within the range of data, the number of extremes and the num-
ber of zero crossings are equal to or vary by one; (2) the envelope
defined by the local maxima and the envelope defined by the local
minima shall be symmetric with respect to the mean. A pair of
white Gaussian noises is added to the original signal in CEEMD
method, respectively. For a signal sðtÞ, the CEEMD method can be
described as the following steps [30].

1. Add a pair of white Gaussian noises to sðtÞ, producing two sig-

nals as follows:

Fig. 5. Different parts of the experimental setup of the beam structure. s1 ðtÞ ¼ sðtÞ þ nðtÞ ð16Þ

between the FE model and the experimental model is that the FE s2 ðtÞ ¼ sðtÞ  nðtÞ ð17Þ
model is excited only from one point at the node No. 8 with ran-
dom excitation (see Fig. 2). where n(t) is the white Gaussian noise.
6 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402

2. Decompose s1 ðtÞ and s2 ðtÞ by Empirical Mode Decomposition stage of training, which results in a decreased network [37]. In
(EMD) [31] and obtain IMF 1i from those positive mixtures and order to normalize the data inside the network, the batch normal-
IMF2i from those negative mixtures. ization (BN) layer is used [9,38]. The BN transformation is given as
3. Repeat the above steps several times and obtain two sets of follows [36]:
ensemble IMFs as follows: ðl1Þ
lB
^ðl1Þ ¼ ypffiffiffiffiffiffiffiffiffi
y 2 ðrB þe
X
n ð23Þ
IMF1 ¼ IMF1i ð18Þ ^ðl1Þ þ bðlÞ
zðlÞ ¼ cðlÞ y
i¼1
where, yðl1Þ is the input vector to the BN layer, zðlÞ is the output
X
n
response related to a neuron in layer l, lB ¼ E ½yðl1Þ ,
IMF2 ¼ IMF2i ð19Þ
i¼1
r2B ¼ var½yðl1Þ , e a small constant for numerical stability, cðlÞ and
bðlÞ are respectively the parameters of scale and shift which
obtained by learning. After each layer, an activation function is
applied [36]. In this study, 2 types of activation functions are used,
4. The final IMFs is the ensemble of both the IMFs with positive Relu and Softmax. The Relu function, as defined in Eq. (24), is used
and negative noises: as the activation function in the convolutional layers and has the
ability to apply nonlinearity and sparseness to the network
IMF 1 þ IMF 2
IMF ¼ ð20Þ structure.
2 
d if d > 0
RðdÞ ¼ ð24Þ
0 otherwise
5. Finally, the original signal sðtÞ can be reconstructed by the fol- The probable distribution of the output classes can be calcu-
lowing formula: lated by Softmax activation function. Therefore, the Softmax func-
tion is used in the last FC layer and defined as [36]:
X
n
sðtÞ ¼ ci ðtÞ þ rn ðtÞ ð21Þ e di
i¼1 rðdÞi ¼ Pk for i ¼ 1; :::k and d ¼ ðd1 ; :::; dk Þ 2 Rk ð25Þ
dj
j¼1 e
where ci ðtÞ is the i-th IMF and rn ðtÞ is the n-th residue.
where d is the input vector and the output values rðdÞ is between 0
2.4. Frequency domain decomposition (FDD) and 1 and their sum is equal to 1.

FDD is an Output-only Modal Analysis (OMA) method which is a 3. Proposed damage detection method
common and accurate technique for identifying the modal param-
eters of the structures and mechanical systems [32,33]. In this In this section, procedure of the proposed method for damage
method, only the measured output responses of the mechanical detection of the experimental beam model under complex working
systems are required for analyzing the characteristics of the sys- load conditions with two random excitations (for more realistic
tem; therefore, the certain forces or excitations are not necessary assumptions) is explained. Fig. 6 displays the block diagram of
requirements in this method. The FDD technique begins with the the proposed damage detection method at a glance. Details of each
basic premise that the characteristics of the system can be stage are explained in the following sections.
acquired by Singular Value Decomposition (SVD) of the Power
Spectral Density (PSD) matrix of measured output responses.
3.1. Dataset
Extracting the peaks of the associated SVD diagram achieves the
natural frequencies of the related mechanical system [32,33]. The-
Generally, in order to investigate the damage detection ability
oretically, the PSD function Gyy ðxÞ can be decomposed as the sin-
P of any method, one may deliberately create different damages in
gular value matrix ðxÞ and the corresponding unitary matrix
a structure or any mechanical system. In this research, five states
UðxÞ as follows [33]:
are considered for both FE and experimental models, including four
X
Gyy ðxÞ ¼ UðxÞ ðxÞ UH ðxÞ ð22Þ damage states and a healthy state; the damage states are created
by adding the lumped masses with different severities to the dif-
where Gyy ðxÞ is the PSD matrix for output responses, and the terms ferent elements of the structure. Consequently, different investi-
P
ðxÞ and UðxÞ are the singular value and unitary matrices gated states included: healthy (Class 1), adding lumped mass
obtained by SVD of Gyy ðxÞ. with severity of 0.1 kg in element No. 3 (Class 2), adding lumped
mass with severity of 0.2 kg in element No. 3 (Class 3), adding
2.5. Deep convolutional neural network (DCNN) lumped mass with severity of 0.1 kg in element No. 8 (Class 4),
and adding lumped mass with severity of 0.2 kg in element No. 8
The Deep Convolutional Neural Network (DCNN) is an improved (Class 5) as described in Table 1. The additional masses 0.1 kg
replacement to the traditional neural network that is very effective and 0.2 kg are almost 20% and 40% of the mass of the structure,
and develops classification methods in the field of machine vision respectively. Adding a lumped mass to the beam allows to calcu-
[34]. There are 2 phases to learn in DCNN; feed forward phase and late the shift of the natural frequencies and the change of the mode
back propagation (BP) [35]. DCNN consists of three main layers, shapes at the measurement location. The created experimental
namely, convolutional, pooling and fully connected (FC) layers beam model for Class 3 is showed in Fig. 7. In the proposed method
[34,36]. The output of convolution layer is called the feature map- in this paper, the FE beam model under a random excitation force
ping. In this research, the max-pooling layer was used which at node No. 8 (which shall be referred to as ‘‘simulated simplified
selects only the maximum values in each feature map. To avoid beam model” in this paper) and experimental healthy state are
the overfitting, the dropout technique is used; therefore, according used for training data. Since the working load conditions are
to a probability, each neuron is thrown out of the network at each repeatedly changed during the real operating condition, the exper-
 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402 7

Fig. 6. The block diagram of the proposed damage detection method.

MeðdÞ ¼ MeðuÞ þ M 1 :ðHT ðxÞ:HðxÞÞjx ¼ 0 þ M 2 :ðHT ðxÞ:HðxÞÞjx ¼ 1

Table 1
M1 ¼ M 2 ¼ M2add
Description of different damage Classes in both FE and experimental models.
ð26Þ
Classes Considered states
Class 1 Healthy where Me is the mass matrix of each element and subfields u and d
Class 2 Damaged (Added lumped mass with severity 0.1 kg in element 3) show healthy and damage state, respectively. M add is the added
Class 3 Damaged (Added lumped mass with severity 0.2 kg in element 3)
Class 4 Damaged (Added lumped mass with severity 0.1 kg in element 8)
lumped mass and x is the added mass distance in each element.
Class 5 Damaged (Added lumped mass with severity 0.2 kg in element 8)

3.2. FE model updating and preprocessing

imental beam is exited from two different nodes for more realistic In the proposed method, each vibration signal for each of dam-
assumptions. Therefore, the experimental beam model under two age state has been normalized by normalization of min–max [10].
random excitation forces at nodes No. 5 and No. 8 (which shall Generally, when dynamic characteristics of the FE and real models
be referred to as ‘‘experimental complex beam model” in this are compared with each other, some differences are found due to
paper) is used for testing data. In order to simulate damage uncertainties such as modeling errors, measurement errors, vary-
states in the simulated simplified beam model, mass increase of ing loading conditions and environmental noises that may cause
M 1 and M 2 for each element is obtained from the following the false alarms for the damage detection process; therefore,
equation [25]: updating the initial FE model is essential to minimizing the model

Fig. 7. The experimental beam model for Class 3.

8 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402

error based on the results of real model and adjusting some param- where X1 and X2 are the vectors of the signals that correlation coef-
eters of the model. In the present study, FE beam model is devel- ficients are calculated for them. lX1 and lX2 are the means of these
oped and updated based on the natural frequencies of the vectors. After selecting the proper IMFs, the original signals are
experimental healthy model. The updating is carried out by alter- reconstructed using these IMFs in the both models,
ing the Young’s modulus of the FE model in such a manner that
the natural frequencies of the FE healthy model matched with 3.4. Proposed DCNN
the natural frequencies of experimental healthy model
[18,20,39,40]. In the following stages, the extracted information The proposed DCNN consists of two parts: first, the convolu-
from the updated model and the experimental healthy model are tional layers, which are designed to extract the features, and sec-
used as training data for damage detection of the experimental ond, the FC layers, which are used for accessing the output layer.
model. Using the cross-library in Python programming language (Keras
In the proposed method, the Data Augmentation (DA) based on framework), the proposed DCNN is implemented. One convolu-
overlap technique is used to increase the number of samples [9]. tional layer (with non-linear Relu function and dropout, max-
Increasing the number of samples increases accuracy and prevents pooling and BN layers) is designed to achieve excellent perfor-
overfitting. In this method, rectangular windows with determined mance; and the same substructure is repeated four times. Since
duration and overlap are used in such a way that the number of the max-pooling operation always reduces the size of the feature
samples increases for each of the damage states. The number of map, so the kernel number increases along with the depth to retain
samples and length of epochs are carefully chosen to achieve the sufficient information about the feature map. Wide kernels
maximum accuracy rate, and finally, 478 and 118 samples with (128  1) are chosen to be design in the first convolutional layer
length of 1904 (sampling points) are assigned to each state of the by taking into account that the wide kernels in the first convolu-
simulated simplified beam model and experimental complex beam tional layer are able to better eliminate the high frequency noises
model, respectively. The DA process is depicted for one state of the compared to the small kernels. Small kernels with the size of
simulated simplified beam model in Fig. 8; for instance, any vibra- 3  1 are used in the next convolutional layers, which result in bet-
tion signal for each of the damage states with length of 409,600 can ter representation of the input features. The step size for the first
provide 478 training samples with length of 1904. Total number of convolutional layer is set to 8 which can increases the speed of
samples assigned to each state of the simulated simplified beam training proposed deep network. More details are presented in
model and experimental complex beam model is shown in Table 2. Table 3, and the architecture of the proposed DCNN is shown in
Fig. 9.
3.3. Proper IMF selection and signal reconstruction Total number of generated samples for training and testing data
is 2390 and 590, respectively, which 60 samples of testing data are
Recorded vibration signals from mechanical systems are always used for validation. All of the hyper-parameters of DCNN structure
exposed to additional undesirable information. So, some parts of are carefully adjusted to obtain the best convergence rate. Finally,
the signals that are not related to the nature of the system can the training process is performed by cross-entropy cost function
be removed using the CEEMD method. To proper IMFs selection, and Adam optimizer [9,36] with learning rate of 0.0001 and batch
dynamic responses of the simulated simplified beam model and size 10. After training the proposed DCNN, the proposed method is
experimental healthy state are decomposed into the IMFs using evaluated using 530 samples of the testing data. The samples allo-
the CEEMD method. Then, the IMFs that have the highest correla- cation used in the proposed method for 2 phases of training and
tion coefficient value are selected [30]. The correlation coefficient testing is shown in Fig. 10.
is a parametric method for large amount of data ðNÞor the data
with normal distribution [41] and is obtained from the following 4. Application of the proposed method for damage detection of
equation [42]: the experimental beam structure
PN
ðX 1i lX1 ÞðX 2i lX2 Þ
g ¼ PN i¼1

1=2 PN
1=2 In this section, the results of the proposed method for damage
ðX 1i lX1 Þ ðX2i lX2 Þ
i¼1 i¼1
ð27Þ detection of the experimental complex beam model are presented.
PN PN
Analyses are performed on a PC laptop with 8 GB of RAM and
lX1 ¼ N1 X 1i lX2 ¼ N1 X 2i
i¼1 i¼1 1.6 GHz Core i5 CPU. By using the proposed trained DCNN (based
on the FE model under simple working load conditions and the real

Fig. 8. DA based on overlap technique for one state of the simulated simplified beam model.
 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402 9

Table 2
Total number of samples for the simulated simplified beam model and experimental complex beam model.

Class 1 Class 2 Class 3 Class 4 Class 5

No. of samples of simulated simplified beam model 478 478 478 478 478
No. of samples of experimental complex beam model 118 118 118 118 118

Table 3
Details of the proposed DCNN architecture.

Number Layer type Kernel size and step size Number of kernels Output value Padding
1 Convolution1 128  1/8  1 16 33  16 Yes
2 Pooling 1 2  1/2  1 16 16  16 No
3 Convolution2 3  1/1  1 32 16  32 Yes
4 Pooling 2 2  1/2  1 32 8  32 No
5 Convolution3 3  1/1  1 64 8  64 Yes
6 Pooling 3 2  1/2  1 64 4  64 No
7 Convolution4 3  1/1  1 64 4  64 Yes
8 Pooling 4 2  1/2  1 64 2  64 No
9 Convolution5 3  1/1  1 64 2  64 Yes
10 Pooling5 2  1/2  1 64 1  64 No
17 Fully-Connected (FC) 100 1 100
16 Softmax 5 1 5

Fig. 9. Architecture of the proposed DCNN.

Fig. 10. The samples allocation to be used in the proposed method for 2 phases of training and testing.

healthy structure), different real data are inputted into the pro- 4.1. Verification and validation of the FE model based on experimental
posed DCNN to detect the different damage states of the structure. tests
This section is divided into the following subsections: (1) Verifica-
tion and validation of the FE model based on experimental tests; To evaluating the accuracy of the FE model results, the natural
(2) IMF selection and vibration signal reconstruction; (3) Damage frequencies of the beam model are obtained using different numer-
detection of the experimental complex beam model. ical and experimental methods and compared with each other. In
10 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402

order to organize the provided information, and making it easy for 4.1.2. Numerical modal analysis (NMA) based on FE model and
the readers to follow the performed procedure, this subsection is eigenvalues method
divided into the following sub-subsections: (1) The modal param- In the NMA of the beam structure based on FE model, the nat-
eters are obtained through the experimental modal tests based on ural frequencies of the structure are extracted by Eigenvalues
Experimental Modal Analysis (EMA) method; (2) The modal method [44]. The first six natural frequencies of the structure in
parameters are obtained using the Numerical Modal Analysis the desired frequency range using the NMA are presented in
(NMA) based on FE model and Eigenvalues method; (3) The modal Table 4. As can be seen, different damage states affect the natural
parameters are obtained experimentally and numerically using the frequencies in each mode number.
OMA method; (4) Comparison of the modal parameters obtained
by the mentioned methods.
4.1.3. Numerical and experimental modal analyses based on OMA
method
4.1.1. Experimental modal tests based on EMA method As stated in Section 2.4, the FDD method requires only the mea-
In the EMA method, structure is excited by certain forces (as sured output responses of structure. In this section, the FDD
inputs) and the response vibration signals are recorded (as out- method is used to extract raw frequency data of the structure. In
puts); therefore, the structural modal parameters are extracted FE beam model, the matrix Cb is obtained by inserting proportional
   
from the Frequency Response Function (FRF) based on the input damping constants a and b into the Eq. (8). The values of a and b
and output. The FRF can be expressed as follows [43]: can be obtained by replacing the natural frequency values x 1
and x 2 (from NMA) and damping values n1 and n2 (from EMA
Sxx ðxÞ method) in Eqs. (9) and (10). The frequency and damping val-
H ðxÞ ¼ ð28Þ uesx 1 , x 2 , n1 and n2 are obtained 35.81 Hz, 143.28 Hz, 0.005
Sxf ðxÞ
and 0.005, respectively. Fig. 13 shows a sample of normalized
vibration signal of simulated simplified beam model and experi-
where Sxx ðxÞ and Sxf ðxÞ are the PSD and Cross Spectral Density mental complex beam model for different states in time domain
(CSD). Subscripts x and f represents the output and input signals, and FDD domain for working frequency range from 0 Hz to
respectively. Natural frequencies and damping ratios can be 1600 Hz.
extracted from any FRF. Fig. 11 shows the FRFs for two accelerom- According to the results shown in Fig. 13(b), it can be seen that
eters I and II mounted on the experimental healthy structure that the SVD diagrams of both models predict approximately similar
excited only by the shaker II (see Fig. 4) in logarithmic scale and fre- behavior, but the uncertainties such as modeling errors, measure-
quency range from 0 Hz to 1600 Hz. As can be seen, the accelerom- ment errors, varying loading conditions and environmental noises
eters mounted on structure have almost identical natural cause the difference in the amplitude of the peaks and their values.
frequencies. Peaks of the SVD diagram specifies the natural frequencies of the
The FRF and coherence for the experimental healthy structure structure which are brought in the following section.
are showed in Fig. 12, which are obtained using the data measured
from shaker II and accelerometer II. It should be noted that the
coherence is defined as follows [43]: 4.1.4. Comparison of the obtained modal parameters
In order to better investigate the results, errors of the first six
  natural frequencies for the healthy structure in the desired fre-
Sxf ðxÞ2
c2 fx ðxÞ ¼ ð29Þ quency range using the numerical and experimental methods are
Sff ðxÞSxx ðxÞ shown in Table 5. As can be seen from Table 5, errors of the
The value of 1 for this parameter indicates that the output is obtained 1st, 2nd, 3rd, 4th, 5th and 6th natural frequencies from
derived from the input, and value of zero indicates that the input the FE simulated model based on the Eigenvalues method with
does not have any effect on the output. Peaks of the FRF diagram respect to the obtained natural frequencies from the experimental
specifies the natural frequencies of the structure which are brought model based on the EMA method are almost 1.15%, 8.71%, 5.69%,
in Section 4.1.4. 10.92%, 11.77% and 10.27%, respectively. Also, errors of the
obtained 2nd and 3rd natural frequencies from the simulated sim-
plified beam model based on the OMA method with respect to the
obtained natural frequencies from the experimental complex beam
model based on the OMA method are almost 8.54% and 9.55%,
respectively; in addition, it can be seen that the obtained 4th,
5th and 6th natural frequencies from the simulated simplified
beam model based on the OMA method are almost equivalent to
the natural frequencies of the experimental complex beam model
based on the OMA method. The comparison of the natural frequen-
cies of the experimental model and those obtained from FE simu-
lated model is carried out by calculating the relative error [45]:
 
f  f FE 
Exp
Error ¼  100 ð30Þ
f Exp

where f Exp is the natural frequency obtained from experimental

model, and f FE is the corresponding natural frequency obtained
from FE simulated model. Comparison of dominant natural frequen-
Fig. 11. FRFs for two accelerometers mounted on the experimental healthy cies shows that the FE model is accurate enough to be representa-
structure. tive of the dynamic behavior of the structure.
 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402 11

Fig. 12. FRF and coherence function for the experimental healthy structure using the data measured from shaker II and accelerometer II. (a) FRF. (b) coherence function.

Table 4
The first six natural frequencies of the structure using the numerical modal analysis.

Mode No. Healthy (Class 1) Damaged (Class 2) Damaged (Class 3) Damaged (Class 4) Damaged (Class 5)
1 35.81 33.76 29.62 30.79 23.85
2 143.28 124.1 102.71 131.98 122.04
3 311.47 286.11 264.33 320.92 317.88
4 573.72 548.42 535.02 511.73 473.82
5 867.85 909.03 901.24 872.17 859.63
6 1196.5 1186.6 1175.8 1194.8 1180.5

4.2. IMF selection and vibration signal reconstruction nearly 89% while the accuracy of the proposed DCNN with fre-
quency data of vibration signals is approximately 86%. This shows
According to the principles presented in Section 3.3, the corre- that the proposed DCNN performs better to detect the different
lation coefficient average of all samples with their original signals damages when using raw frequency data of reconstructed vibra-
(correlation coefficient between original signal xðtÞ and each IMF) tion signals as input. Detection accuracy of the proposed DCNN
for IMF1 to IMF12 of the simulated simplified beam model and with frequency data as input is acceptable with respect to the
experimental complex healthy state are listed in Table 6. uncertainties such as modeling errors, measurement errors, vary-
It is obvious from Table 6 that the correlation coefficient aver- ing loading conditions and environmental noises.
age decreases as the number of IMFs increases. It can be seen that To illustrate the performance of the DCNN with various data
IMF1 to IMF7 values are significantly higher than the other IMFs; types as input, the classification accuracy is obtained using the
therefore, it can be concluded that IMF1 to IMF7 contain more other methods. In this regard, time data and several manual fea-
information from the original signal than the other IMFs. Fig. 14 tures from time and frequency data along with BPNN and SVM
shows IMF1 to IMF12 for one sample of the healthy state of the are selected as the comparative methods [21–24]. The Gaussian
simulated simplified beam model and experimental complex beam Radial Basis Function (RBF) is used as the kernel function of the
model using the FDD method. SVM, and the grid search method is used to optimize the kernel
According to the diagrams presented in Fig. 14, the natural fre- parameters. The BPNN architecture consists of a single hidden
quencies of the structure are gradually disappeared as the number layer which Sigmoid activation function is used in this layer. In
of IMFs increases and cannot be distinguished. Therefore, consider- order to achieve better results from BPNN and DCNN models, their
ing the information presented in Table 6 and Fig. 14, IMF1 to IMF7 hyper-parameters are adjusted according to different data types.
are selected as effective components and IMF8 to IMF12 are aban- The parameters of Root Mean Square (RMS), skewness, crest factor,
doned. After selecting the proper IMFs, the original signals of all variance, kurtosis, minimum, maximum, and mean are chosen as
samples are reconstructed using these IMFs. the manual features in the time domain (time features) [7,46,47].
The dominant natural frequencies [48] in desired frequency range
4.3. Damage detection of the experimental complex beam model are selected as the manual features in the frequency domain (fre-
quency features). The testing accuracy of the different methods
In the last stage, the raw frequency data (not the dominant nat- based on the feature learning from raw data and the manual fea-
ural frequencies) of reconstructed vibration signals of the experi- tures are presented in Table 8 and compared in Fig. 15, where
mental healthy model and different states of the simulated the result of the proposed DCNN with the raw frequency data
simplified beam model are used to train the proposed DCNN, and (not the dominant natural frequencies) in desired frequency range
the raw frequency data of reconstructed vibration signals of differ- as input, namely the proposed method, is marked in bold in Table 8.
ent states of the experimental complex beam model are applied to The architecture presented in Table 3 is used in the proposed
test the proposed DCNN. Raw frequency data is extracted using method.
FDD method in the working frequency range from 0 Hz to In the case of two data types with feature learning, the fre-
1600 Hz. Table 7 shows the testing accuracy of the proposed DCNN quency data shows better results than time data using the DCNN.
with frequency data of reconstructed vibration signals and vibra- Table 8 and Fig. 15 show that the frequency data with the proposed
tion signals in desired frequency range as input in order to damage DCNN achieves the test accuracy of 89%, while the time data with
detection of the experimental complex beam model. DCNN can only obtain around 59%. The reason for this is in the
According to this table, the testing accuracy of the proposed characteristic of the convolutional architecture of DCNN, which is
DCNN with frequency data of reconstructed vibration signals is able to overcome the small frequency shift of damage patterns
12 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402

(a)

(b)

Fig. 13. A sample of normalized vibration signal of simulated simplified beam model and experimental complex beam model for different states. (a) Time domain (b) FDD
domain.

by detecting local conjunction features in the frequency domain. suitable to adaptively learn hierarchical features and capture par-
On the other hand, the detection of these local conjunction features tial invariance along the frequency axis [1]. In addition, the fre-
with DCNN may be affected by a slight implicit label shift and a quency data show how their constitutive components are
smaller range of time domain data. This makes the model more distributed at discrete frequencies and can provide clear informa-
 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402 13

Table 5
Comparison of the first six natural frequencies for the healthy structure using different methods.

Mode No. Natural Frequency (Hz) Error (%)

Experimental Model FE simulated Model FE compared with Experimental
Based on EMA Based on OMA Based on Eigenvalues Based on OMA Eigenvalues-EMA OMA-OMA
1 35.4 – 35.81 – 1.15% –
2 131.8 122.9 143.28 133.4 8.71% 8.54%
3 294.7 265.8 311.47 291.2 5.69% 9.55%
4 517.2 471 573.72 471 10.92% 0%
5 776.4 880.6 867.85 880.6 11.77% 0%
6 1085 1188 1196.5 1188 10.27% 0%

Table 6
The correlation coefficient average of all samples with their original signals for IMF1 to IMF12 of the simulated simplified beam model and experimental complex healthy state.

IMF1 IMF2 IMF3 IMF4 IMF5 IMF6

Healthy state (simulated simplified beam model) 0.89 0.40 0.38 0.29 0.15 0.20
Class 2 (simulated simplified beam model) 0.79 0.41 0.29 0.28 0.17 0.31
Class 3 (simulated simplified beam model) 0.70 0.21 0.27 0.28 0.21 0.18
Class 4 (simulated simplified beam model) 0.82 0.27 0.20 0.25 0.33 0.19
Class 5 (simulated simplified beam model) 0.75 0.24 0.32 0.45 0.46 0.14
Healthy state (experimental complex beam model) 0.74 0.52 0.37 0.44 0.16 0.15
IMF7 IMF8 IMF9 IMF10 IMF11 IMF12
Healthy state (simulated simplified beam model) 0.10 0.07 0.03 0.00 0.00 0.00
Class 2 (simulated simplified beam model) 0.12 0.05 0.09 0.007 0.00 0.00
Class 3 (simulated simplified beam model) 0.21 0.07 0.02 0.01 0.00 0.01
Class 4 (simulated simplified beam model) 0.11 0.09 0.08 0.03 0.01 0.00
Class 5 (simulated simplified beam model) 0.13 0.08 0.09 0.03 0.02 0.00
Healthy state (experimental complex beam model) 0.19 0.03 0.01 0.00 0.00 0.02

tion about health conditions [5,49]. Interestingly, the same phe- samples of Class 5 have moderate overlap with samples of Class 4.
nomenon also appears in the DNN based speech analysis [50–52] Additionally, the samples of other classes have a little overlap with
and vibration analysis [1,5,6,53,54]. Comparing performance of each other. This indicates that the proposed DCNN is able to extract
feature learning and manual features shows that feature learning high-level features from raw frequency data and relatively has a
from raw frequency data with proposed DCNN, provides better good performance in discriminating of different classes.
results than manual features, with an increase of accuracy around
9%. This result is significantly correlated with the unique architec-
ture of proposed DCNN, which can automatically extract the useful 5. Conclusion
features of the frequency data layer by layer for classification. In
addition, obtaining of manual features requires expertise and prior In this paper, a novel damage detection method is proposed to
knowledge, while feature learning from raw frequency data is less learn features from raw frequency data of the FE model and real
dependence on expertise knowledge. While proposed DCNN has a healthy state, in order to damage detection of a real system in pres-
better result with feature learning from raw frequency data, all the ence of various uncertainties. A developed DCNN with training
tested models, including DCNN, BPNN and SVM provide similar interference and customized architecture is designed to learn
results with manual features. This indicates that the DCNN cannot damage-sensitive features directly from raw frequency data (not
achieve much more improvements in damage detection than tradi- the dominant natural frequencies). In order to train the proposed
tional methods without the ability of feature learning. DCNN, raw frequency data of the FE model and real healthy state
Fig. 16. shows the accuracy of the proposed method for damage are used. Then, raw frequency data of the real model is used to test
detection of the experimental complex beam model in 500 itera- the proposed DCNN. In this method, only data of real healthy sys-
tions. As shown from Fig. 16, the accuracy of the proposed method tem is used to updating the FE model and minimizing the errors.
reaches 89% for validation data. To more investigate the perfor- Some parts of the vibration signals which are not related to the
mance of the proposed method, the confusion matrix and ROC dia- nature of the system are eliminated using CEEMD method. The
gram are provided in Fig. 17. raw frequency data is obtained using the FDD method. Two main
Lastly, visualization of the feature representations of all the interferences affect the proposed DCNN, the wide kernel in the first
testing samples extracted from the raw time signal and the last convolution layer and small kernels in the remaining convolutional
FC layer via t-SNE method [9,10,16,55,56] is given in Fig. 18. The layers. The first interference is more able to eliminate the high fre-
feature representations are depicted as a scatter diagram with dis- quency noises compared to small kernels, and the second interfer-
tinct colors, which express different classes. According to Fig. 18, it ence can better represent the input features; and therefore,
can be seen from the visualization of the raw time data that it is improves the overall performance of the network. The effective-
difficult to discriminate between samples of different classes and ness of the proposed method is validated using the vibration data
it can be said that raw time features are low-level features. In addi- of an experimental beam.
tion, as can be seen from the visualization of the last FC layer, sam- Time data and frequency data along with DCNN are checked
ples of Class 4 have moderate overlap with samples of Class 2, and and compared with the manual features and two intelligent meth-
14 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402

Fig. 14. IMF1 to IMF12 for one sample of the healthy state of the simulated simplified beam model and experimental complex beam model using the FDD method.

Table 7 DCNN obtains the highest accuracy with frequency data compared
The testing accuracy of the proposed DCNN with different inputs. to time data, this means that the DCNN is more appropriate for
Methods Feature learning from raw data Accuracy % learning features from frequency data, and the same trend is also
Proposed DCNN Frequency data of vibration signals 86 ± 4.1 apparent in DNN based speech and vibration analyses.
Proposed DCNN Frequency data of reconstructed vibration 89 ± 4.3 In the proposed method, all parameters of the proposed DCNN
signals are carefully adjusted in order to achieve the best rate of conver-
gence. The manual selection of the parameters of the deep learning
method can take a lot of trials and errors to find the best one.
Therefore, investigating the effective methods to optimize the
parameters of deep learning methods is necessary. In addition,
ods, BPNN and SVM. The results show that after training using data combinations of various deep learning architectures can be used
sets based on FE model and real healthy state, the proposed to improve the accuracy of damage detection. Meanwhile, only
method is capable to learn features adaptively and obtain convinc- time data and the frequency data were collected to be processed
ing detection results for the real beam model with varying dam- by 1D-CNN. In future works, the performance of 2D models of deep
ages. Compared to manual features, the proposed method learning methods for 2D data processed through time-frequency
increases classification accuracy by approximately 9% and as well analysis techniques such as Wavelet Transform (WT) and
as less dependence on expertise knowledge. Simultaneously, the Wigner-Ville Distribution (WVD) can be studied. Finally, beam

Table 8
The testing accuracy of proposed method compared with other methods.

Methods Feature learning from raw data Manual features

Time data Frequency data Time features Frequency features
SVM 35 ± 1.4 74 ± 2.5 54 ± 1.5 79 ± 2.3
BPNN 34 ± 2.6 77 ± 3.6 53 ± 1.8 80 ± 2.5
DCNN 59 ± 2.8 89 ± 4.3 53 ± 2.2 79 ± 1.2
 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402 15

Fig. 15. The testing accuracy of different methods for damage detection of beam structure.
Accuracy (%)

Number of Iteraons

Fig. 16. Accuracy of the proposed method for damage detection of the experimental complex beam model.

Fig. 17. Confusion Matrix and ROC diagram based on proposed method (a) Confusion Matrix, (b) ROC diagram.
16 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402

Fig. 18. Feature visualization via t-SNE: feature representations for all testing samples extracted from (a) the raw time signal, (b) the last FC layer.

structure is a simple mechanical system, and the proposed method [16] Lin YZ, Nie ZH, Ma HW. Structural damage detection with automatic feature-
extraction through deep learning. Comput Aided Civil Infrastruct Eng 2017;32
can be developed and generalized for further complex mechanical
(12):1025–46.
systems under varying uncertainties. [17] Pathirage CSN, Li J, Li L, Hao H, Liu W, Ni P. Structural damage identification
based on autoencoder neural networks and deep learning. Eng Struct
2018;172:13–28.
[18] Mojtahedi A, Yaghin ML, Hassanzadeh Y, Ettefagh M, Aminfar M, Aghdam A.
Declaration of Competing Interest
Developing a robust SHM method for offshore jacket platform using model
updating and fuzzy logic system. Appl Ocean Res 2011;33(4):398–411.
The authors declare that they have no known competing finan- [19] Teng S, Chen G, Liu G, Lv J, Cui F. Modal strain energy-based structural damage
detection using convolutional neural networks. Appl Sci 2019;9(16):3376.
cial interests or personal relationships that could have appeared
[20] Ding Z, Li J, Hao H, Lu Z-R. Structural damage identification with uncertain
to influence the work reported in this paper. modelling error and measurement noise by clustering based tree seeds
algorithm. Eng Struct 2019;185:301–14.
[21] Beale HD, Demuth HB, Hagan M. Neural network design, Pws, Boston; 1996.
[22] Hajnayeb A, Ghasemloonia A, Khadem S, Moradi M. Application and
References
comparison of an ANN-based feature selection method and the genetic
algorithm in gearbox fault diagnosis. Expert Syst Appl 2011;38(8):10205–9.
[1] Jing L, Zhao M, Li P, Xu X. A convolutional neural network based feature [23] Santos P, Villa LF, Reñones A, Bustillo A, Maudes J. An SVM-based solution for
learning and fault diagnosis method for the condition monitoring of gearbox. fault detection in wind turbines. Sensors 2015;15(3):5627–48.
Measurement 2017;111:1–10. [24] HosseinAbadi HZ, Amirfattahi R, Nazari B, Mirdamadi HR, Atashipour SA.
[2] Cofre-Martel S, Kobrich P, Lopez Droguett E, Meruane V. Deep convolutional GUW-based structural damage detection using WPT statistical features and
neural network-based structural damage localization and quantification using multiclass SVM. Appl Acoust 2014;86:59–70.
transmissibility data. Shock Vib 2019;2019. [25] Weaver Jr W, Johnston PR. Structural dynamics by finite elements. Englewood
[3] Wang X, Jiao J, Yin J, Zhao W, Han X, Sun B. Underwater sonar image Cliffs (NJ): Prentice-Hall; 1987.
classification using adaptive weights convolutional neural network. Appl [26] Chowdhury I, Dasgupta SP. Computation of Rayleigh damping coefficients for
Acoust 2019;146:145–54. large systems. Electron J Geotech Eng 2003;8:1–11.
[4] Bao Y, Tang Z, Li H, Zhang Y. Computer vision and deep learning–based data [27] Wu S, Law S. Vehicle axle load identification on bridge deck with irregular road
anomaly detection method for structural health monitoring. Struct Health surface profile. Eng Struct 2011;33(2):591–601.
Monit 2019;18(2):401–21. [28] Varahram S, Jalali P, Sadeghi MH, Lotfan S. Experimental study on the effect of
[5] Jia F, Lei Y, Lin J, Zhou X, Lu N. Deep neural networks: a promising tool for fault excitation type on the output-only modal analysis results. Trans FAMENA
characteristic mining and intelligent diagnosis of rotating machinery with 2019;43(3):37–52.
massive data. Mech Syst Sig Process 2016;72:303–15. [29] Torres ME, Colominas MA, Schlotthauer G, Flandrin P. A complete ensemble
[6] Rafiei MH, Adeli H. A novel unsupervised deep learning model for global and empirical mode decomposition with adaptive noise. In: 2011 IEEE
local health condition assessment of structures. Eng Struct 2018;156:598–607. international conference on acoustics, speech and signal processing
[7] Jing L, Wang T, Zhao M, Wang P. An adaptive multi-sensor data fusion method (ICASSP); 2011. IEEE. pp. 4144–4147.
based on deep convolutional neural networks for fault diagnosis of planetary [30] Li M, Wang H, Tang G, Yuan H, Yang Y. An improved method based on CEEMD
gearbox. Sensors 2017;17(2):414. for damage diagnosis of rolling bearing. Adv Mech Eng 2014;6:676205.
[8] Fallahian M, Khoshnoudian F, Meruane V. Ensemble classification method for [31] Huang NE et al. The empirical mode decomposition and the Hilbert spectrum
structural damage assessment under varying temperature. Struct Health Monit for nonlinear and non-stationary time series analysis. Proc Roy Soc Lond Ser A
2018;17(4):747–62. Math Phys Eng Sci 1998;454(1971):903–95.
[9] Zhang W, Li C, Peng G, Chen Y, Zhang Z. A deep convolutional neural network [32] Brincker R, Zhang L, Andersen P. Modal identification of output-only systems
with new training methods for bearing fault diagnosis under noisy using frequency domain decomposition. Smart Mater Struct 2001;10(3):441.
environment and different working load. Mech Syst Sig Process [33] Yi J-H, Park J-S, Han S-H, Lee K-S. Modal identification of a jacket-type offshore
2018;100:439–53. structure using dynamic tilt responses and investigation of tidal effects on
[10] Chen Y, Peng G, Xie C, Zhang W, Li C, Liu S. ACDIN: bridging the gap between modal properties. Eng Struct 2013;49:767–81.
artificial and real bearing damages for bearing fault diagnosis. [34] Goodfellow I, Bengio Y, Courville A. Deep learning. Cambridge, MA: MIT Press;
Neurocomputing 2018;294:61–71. 2016.
[11] Ahmed H, Wong MD, Nandi AK. Intelligent condition monitoring method for [35] Hung S-L, Adeli H. Parallel backpropagation learning algorithms on Cray Y-
bearing damages from highly compressed measurements using sparse over- MP8/864 supercomputer. Neurocomputing 1993;5(6):287–302.
complete features. Mech Syst Sig Process 2018;99:459–77. [36] Mousavi Z, Rezaii TY, Sheykhivand S, Farzamnia A, Razavi S. Deep
[12] Zhao R, Yan R, Chen Z, Mao K, Wang P, Gao RX. Deep learning and its convolutional neural network for classification of sleep stages from single-
applications to machine health monitoring. Mech Syst Sig Process channel EEG signals. J Neurosci Methods 2019:108312.
2019;115:213–37. [37] Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR.
[13] Kiranyaz S, Avci O, Abdeljaber O, Ince T, Gabbouj M, Inman DJ. 1D Improving neural networks by preventing co-adaptation of feature detectors,
convolutional neural networks and applications: a survey, arXiv preprint arXiv preprint arXiv: 1207.0580, 2012
arXiv:1905.03554, 2019. [38] Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by
[14] Abdeljaber O, Avci O, Kiranyaz S, Gabbouj M, Inman DJ. Real-time vibration- reducing internal covariate shift, arXiv preprint arXiv: 1502.03167, 2015.
based structural damage detection using one-dimensional convolutional [39] Mosallam A, Zirakian T, Abdelaal A, Bayraktar A. Health monitoring of a steel
neural networks. J Sound Vib 2017;388:154–70. moment-resisting frame subjected to seismic loads. J Constr Steel Res
[15] Turner M. Stiffness and deflection analysis of complex structures. J Aeronaut 2018;140:34–46.
Sci 1956;23(9):805–23.
 Z. Mousavi et al. / Applied Acoustics 168 (2020) 107402 17

[40] Barton E, Middleton C, Koo K, Crocker L, Brownjohn J. Structural finite element [49] Ng SS, Tse PW, Tsui KL. A one-versus-all class binarization strategy for bearing
model updating using vibration tests and modal analysis for NPL Footbridge– diagnostics of concurrent defects. Sensors 2014;14(1):1295–321.
SHM demonstrator. J Phys Conf Ser 2011;305(1). p. 012105: IOP Publishing. [50] Abdel-Hamid O, Deng L, Yu D. Exploring convolutional neural network
[41] Giampieri E et al. Statistical strategies and stochastic predictive models for the structures and optimization techniques for speech recognition. Interspeech
MARK-AGE data. Mech Ageing Dev 2015;151:45–53. 2013;2013:1173–5.
[42] Bafroui HH, Ohadi A. Application of wavelet energy and Shannon entropy for [51] Deng L, Hinton G, Kingsbury B. New types of deep neural network learning for
feature extraction in gearbox damage detection under varying speed speech recognition and related applications: an overview. In: 2013 IEEE
conditions. Neurocomputing 2014;133:437–45. international conference on acoustics, speech and signal processing. IEEE;
[43] Richardson MH, Formenti DL. Parameter estimation from frequency response 2013. p. 8599–603.
measurements using rational fraction polynomials. In: Proceedings of the 1st [52] Sainath TN, Mohamed A-R, Kingsbury B, Ramabhadran B. Deep convolutional
international modal analysis conference, vol. 1: Union College Schenectady, neural networks for LVCSR. In: 2013 IEEE international conference on
NY; 1982. p. 167–186. acoustics, speech and signal processing. IEEE; 2013. p. 8614–8.
[44] Meirovitch L. Analytical methods in vibrations. New York: Macmillan; 1967. [53] Li C, Sanchez R-V, Zurita G, Cerrada M, Cabrera D, Vásquez RE. Multimodal
[45] Wang Y. Damage assessment in asymmetric buildings using vibration deep support vector classification with homologous features and its
techniques, PhD Thesis, Queensland University of Technology; 2018. application to gearbox fault diagnosis. Neurocomputing 2015;168:119–27.
[46] Fedala S, Rémond D, Zegadi R, Felkaoui A. Contribution of angular [54] Huijie Z, Ting R, Xinqing W, You Z, Husheng F. Fault diagnosis of hydraulic
measurements to intelligent gear faults diagnosis. J Intell Manuf 2018;29 pump based on stacked autoencoders. In: 2015 12th IEEE international
(5):1115–31. conference on electronic measurement & instruments (ICEMI). IEEE; 2015. p.
[47] Sheykhivand S, Rezaii TY, Farzamnia A, Vazifehkhahi M. Sleep stage scoring of 58–62.
single-channel EEG signal based on RUSBoost classifier. In: 2018 IEEE [55] Zhang W, Peng G, Li C, Chen Y, Zhang Z. A new deep learning model for fault
international conference on artificial intelligence in engineering and diagnosis with good anti-noise and domain adaptation ability on raw vibration
technology (IICAIET). IEEE; 2018. p. 1–6. signals. Sensors 2017;17(2):425.
[48] Jamalkia A, Ettefagh MM, Mojtahedi A. Damage detection of TLP and Spar [56] Yang K, Chu R, Zhang R, Xiao J, Tu R. A novel methodology for series arc fault
floating wind turbine using dynamic response of the structure. Ocean Eng detection by temporal domain visualization and convolutional neural
2016;125:191–202. network. Sensors 2020;20(1):162.