Expert Systems With Applications 237 (2024) 121521

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

Dual-source gramian angular field method and its application on fault

diagnosis of drilling pump fluid end
Gang Li a, b, 1, *, Jiaxing Ao a, 2, Jiayao Hu a, 3, Dalong Hu a, 4, Yang Liu a, 5, Zhiqiang Huang a, 6
a
School of Mechatronic Engineering, Southwest Petroleum University, Chengdu 610500, People’s Republic of China
b
Oil and Gas Equipment Technology Sharing and Service Platform of Sichuan Province, Southwest Petroleum University, Chengdu 610500, People’s Republic of China

A R T I C L E I N F O A B S T R A C T

Keywords: Under complex drilling working conditions, it is difficult to completely describe the working state of the drilling
Drilling pump fluid end pump fluid end with a single-source signal to achieve a high fault diagnosis accuracy. This paper proposes to use
Fault diagnosis dual-source signals (vibration signal and strain signal) to describe the working state of the fluid end. Especially, a
Dual-source gramian angular field
dual-source gramian angular field (DS-GAF) method is proposed to fuse vibration-strain signals to provide more
ResNet
Signal fusion
fault information. In DS-GAF method, the dual-source signal is converted to points in spherical coordinates, and
the arccosine of the normalized amplitude of the two signals is assumed as the polar angle and the azimuthal
angle, respectively. Three DS-GAF matrices are generated with the position of the points. These three matrices
are respectively encoded as three colors (red, green, and blue) channels of the image to achieve signal fusion. A
fault diagnosis method based on DS-GAF and ResNet for the drilling pump fluid end is proposed to improve the
fault diagnosis accuracy under multiple working conditions. First, by the DS-GAF method, the vibration-strain
signals of the fluid end are fused and converted into a fault diagnosis image dataset. Second, a fault diagnosis
model based on ResNet is established and the effect of batch size on the performance of the fault diagnosis model
is discussed. Finally, fault diagnosis experiments are carried out on the proposed fault diagnosis model and its
performance is investigated. The results show that the diagnosis method proposed in this paper has the highest
diagnostic accuracy of 98.97% for nine fault types under multiple operating conditions, with a maximum
improvement of 18.91% compared to the original GAF method. The DS-GAF method for signal fusing proposed in
this paper not only improves the accuracy of fluid end diagnosis, but also provides a reference for the signal-
fusion diagnosis of other equipment.

1. Introduction Especially, the suction valves and discharge valves, which are in direct
contact with the drilling fluid, are weakest in the drilling pump fluid
As shown in Fig. 1, the drilling pump is the core equipment of the oil end.
and gas drilling circulation system (Li et al., 2023). The main role of the At present, the fault diagnosis of the drilling pump fluid end is mainly
drilling pump is to transport drilling fluid to the bottom of the well to based on the experience of operators, which has low accuracy and poor
ensure the drilling fluid circulation and the safety and efficiency of the real-time performance. Therefore, there is an urgent need to carry out
drilling operation. It easily causes the failure of the fluid end of the research on data-driven drilling pump fault diagnosis methods to
drilling pump, due to the high working pressure of the drilling pump, as improve the accuracy and efficiency of drilling pump fault diagnosis.
well as the high abrasiveness and corrosiveness of drilling fluid. This is of great significance to ensure the stable and reliable operation of

* Corresponding author at: No. 8 Xindu Avenue, Southwest Petroleum University, Xindu District 610500, Chengdu, Sichuan, China.
E-mail addresses: [email protected] (G. Li), [email protected] (J. Ao), [email protected] (J. Hu), [email protected] (D. Hu), [email protected]
(Y. Liu), [email protected] (Z. Huang).
1
https://orcid.org/0000-0001-9903-9121.
2
https://orcid.org/0000-0002-6334-448X.
3
https://orcid.org/0000-0002-3400-9041.
4
https://orcid.org/0000-0002-2736-0909.
5
https://orcid.org/0000-0003-4906-2280.
6
https://orcid.org/0000-0001-7809-3241.

https://doi.org/10.1016/j.eswa.2023.121521
Received 26 April 2023; Received in revised form 30 August 2023; Accepted 7 September 2023
Available online 15 September 2023
0957-4174/© 2023 Elsevier Ltd. All rights reserved.
G. Li et al. Expert Systems With Applications 237 (2024) 121521

Fig. 1. The drilling pump and the fluid end.

drilling pumps, and to ensure the safe and efficient operation of oil and distillation tower faults with an accuracy of 92.2%. Youcef Khodja et al.
gas drilling. (2020)diagnosed five types of bearing faults by CNN and vibration
In recent years, fault diagnosis methods based on data-driven for spectrum imaging (VSI) with an accuracy of 98.26% under multiple
pumps are rapidly developed. Deng et al. (2017) extracted the energy working conditions. Choudhary et al. (2021) proposed applying thermal
features of each frequency band of the acoustic emission signal by imaging maps of bearings for fault diagnosis. The experimental results of
wavelet packets and composed them into a feature vector. The diag­ six types of bearing fault diagnosis showed that CNN achieved 99.80%
nostic accuracy of the four faults at the drilling pump fluid end was accuracy, which was higher than the accuracy of ANN at 88.67%. Shao
96.56% by the genetic algorithm optimized back propagation (BP) and Kim (2022) combined CNN with DAE and processed features by
neural network. Dutta et al. (2023) used a multiclass support vector correlation alignment (CORAL) to realize unsupervised learning. The
machine (MSVM) to diagnose whether a centrifugal pump was faulty, fault diagnosis accuracy for Case Western Reserve University (CRWU)
with a maximum diagnostic accuracy of 99.8% at 2800 RPM. Tang et al. bearing data sets reached 99.93% with no significant overfitting. Guo
(2022) transformed the vibration signal into a time–frequency map by et al. (2022) applied CNN-LSTM for fault diagnosis of the hydraulic
continuous wavelet transform (CWT) and input a Bayesian-optimized system of the blanket transfer device and achieved an accuracy of
convolutional neural network (CNN) fault diagnosis model to achieve 98.56% for five types of faults with no significant overfitting. Xiao et al.
97.44% accuracy in diagnosing five types of faults in plunger pumps. To (2022) proposed to reconstruct the time domain signal with improved
address the difficulty in diagnosing faults caused by the lack of markers variational mode decomposition (VMD), then converted it into an image
of pumping rod pump motor power curves (MPCs) in practical appli­ by CWT, and finally combined it with CNN for fault diagnosis. The ac­
cations, Hao and Gao (2022) proposed to match the simulated MPCs curacy of fault diagnosis of CRWU bearing data sets was 99.20%. Wang
with the actual MPCs through the mechanism-assisted domain adapta­ et al. (2022) encoded time-domain signals into images by Markov
tion network (MADAN). In their study, an unsupervised learning model transition field and applied CNN for fault diagnosis, with the highest
for six types of faults was achieved with an accuracy of 98.33%. In accuracy of 100% for four types of fault diagnosis for CRWU bearing
drilling operations, with a complex working environment, multiple fault data sets. Ayas and Ayas (2022) constructed a ResNet model for fault
types, and a large amount of noise in the raw signals, it is difficult to diagnosis on CRWU bearing data sets, which achieved an accuracy of
extract fault features from the fluid end of the drilling pump. Therefore, 99.9%. Liang et al. (2022) proposed a rolling bearing fault diagnosis
research on fault diagnosis methods for the drilling pump fluid end has method based on wavelet transform and improved residual network,
been slowly developed. At present, the research on the fault diagnosis which achieved 100% fault diagnosis accuracy for CRWU bearing data
method for drilling pump fluid end is in the early stage of few diag­ sets. Liu et al. (2023) proposed a new deep residual network to diagnose
nosable fault types, low accuracy, and few applicable working wind turbine faults with an accuracy of 99.68%.
conditions. The studies above show that with the development and maturity of
Traditional machine learning methods usually require the extraction deep learning networks, the accuracy of fault diagnosis for a variety of
of fault features manually, leading to inconsistent results of signal pro­ complex machinery is increasing. Two-dimensional CNN provides the
cessing, and causing low fault diagnosis accuracy. Deep learning net­ advantages of simple operation and high accuracy in dealing with multi-
works can extract features of signals automatically without manually classification fault diagnosis problems. However, appropriate methods
processing large amounts of complex data, which simplifies operations of converting the acquired one-dimensional signals into images are
and improves fault diagnosis accuracy. Pookkuttath et al. (2023) required.
detected vibrations during the operation of cleaning robots by a camera. GAF (Cui, Zhong, Zheng, Peng, & Wen, 2022; Ding, Wang, Ding, &
The vibration source was identified by a CNN with an accuracy of Yuan, 2022; Lee, Zhuo, & Le, 2022; Liang, Cao, & Zhao, 2022) can not
93.8%. Based on the current signal and 1D CNN, Mitra et al. (2022) only express the original information of the signal, but also express the
developed a transmission line fault diagnosis method with a diagnostic correlation between each sampling point of the time domain signal.
accuracy of 99.82% and a diagnostic time of 20 ms. Li et al. (2019) Compared with the original signal, the GAF method is used to encode the
combined CNN and auto-encoder (DAE) to diagnose five types of time domain signal into an image, which provides more fault feature

2
G. Li et al. Expert Systems With Applications 237 (2024) 121521

Fig. 2. The procedure of GAF.

information. Therefore, using the GAF method can effectively improve the indicators of the correlation between them. Ignoring vector lengths,
the accuracy of fault diagnosis. However, GAF can only be applied to the the higher correlation between two vectors, the scalar product is closer
encoding of single-source signals, and cannot be used for dual-source to 1, while the opposite is closer to − 1. The Gramian matrix G is
signal fusion. If a GAF-based signal-fusion method can be proposed, it introduced to represent the correlation between vectors.
will help to mine more fault signals. ⎡〈 〉 〈 〉 〈 〉 ⎤
The above studies show that research on fault diagnosis methods for →
x 1, →
x1 →
x 1, →x2 ⋯ →
x 1, →xn
⎢〈 〉 ⎥
the drilling pump fluid end is progressing slowly. The main reason is that ⎢
⎢ → →
〉 〈 〉 〈 ⎥
⎥
→ → → →
the complex working condition of the drilling pump causes a lot of noise G=⎢
⎢ x 2, x 1 x 2 , x 2 ⋯ x 2 , x N ⎥
⎥
in the raw signals, which makes it difficult to extract fault features. ⎢
⎢〈 ⋮ 〉 〈 ⋮ 〉 ⋱ 〈 ⋮ 〉 ⎥
⎥
Moreover, a single-source signal cannot accurately describe the working ⎣ ⎦
→x N, →
x1 →
x N, →x2 ⋯ →x N, →xN
state of the fluid end of the drilling pump. These problems lead to the
low accuracy of fault diagnosis of the fluid end of the drilling pump, few
For time series collected by sensors, each sample point is a scalar
types of faults identified, and few applicable working conditions. Deep
rather than a vector which cannot be directly converted into a Gramian
learning provides an approach to address these problems in fluid end
matrix. Thus, the GAF is introduced. The procedure of GAF is shown in
fault diagnosis of drilling pumps, but an appropriate signal processing
Fig. 2.
method must first be developed.
Firstly, the time series X={xi|i = 1,2,3,…, N} of length N are
This paper proposes to use dual-source signals (vibration signal and
normalized to obtain the normalized amplitude [-1,1].
strain signal) to describe the working state of the fluid end of drilling
pumps, and especially develops a dual-source Gramian angular field (xi − max{X} ) + (xi − min{X} )
xi =
̃
(DS-GAF) method that fuses the vibration signal and strain signal. In the max{X} − min{X}
DS-GAF method, firstly, the arccosine of the normalized amplitude of
Secondly, the time series is converted to polar coordinates. The polar
each sample point for both signals are assumed as the polar angle and
angle θi of each sampling point is the arccosine of the normalized
azimuthal angle, respectively. The ratio of the position of the sampling
amplitude. The radius ri is the ratio between the sequence i of the point
point to the total length of the signal is assumed as the radius. So, the
and the total length N of the time series.
dual-source signal is converted to spherical coordinates. Secondly, three
⎧
DS-GAF matrices are calculated based on the location association be­ ⎪
⎨ θi = arccos̃
xi
tween each point in the coordinates, and three colors (red, green, and
i
blue) channels of the image are encoded with the DS-GAF matrices. ⎪
⎩ ri =
N
Finally, three color channels are combined as an image to achieve signal
fusion. The time series acquired by the sensor is converted to polar co­
This paper is organized as follows. In section 2, the GAF and ResNet ordinates by the above equations. The amplitude information is pre­
are described. In section 3, the proposed DS-GAF method, the process of served by the polar angle θi and the time information by the polar
fault diagnosis method for drilling pump fluid end, and experiments of diameter ri. The points in polar coordinates correspond to the points
data acquisition are introduced in detail. In section 4, the experimental sampled in the time series. There is no loss of information in the time
validations on drilling pump fluid end datasets and gearbox datasets are series.
provided. The effectiveness of DS-GAF is verified. In Section 5, this paper Finally, GAF defines the scalar product of two points xi and xj as the
is concluded. cosine of the sum of the polar angles of the two points in polar co­
ordinates, that is, gij=<xi,xj>=cos(θi + θj). The GAF can be expressed as
2. Theoretical backgrounds ⎡ ⎤
cos(θ1 + θ1 ) cos(θ1 + θ2 ) ⋯ cos(θ1 + θN )
⎢ cos(θ2 + θ1 ) cos(θ2 + θ2 ) ⋯ cos(θ2 + θN ) ⎥
2.1. GAF GAF = gij = ⎢ ⎣
⎥
⎦
⋮ ⋮ ⋱ ⋮
cos(θN + θ1 ) cos(θN + θ2 ) ⋯ cos(θN + θN )
With the rapid development of machine vision in recent years, the
ability of computers to recognize and classify images has subsequently GAF not only retains the information of the raw data but also ex­
increased rapidly. The concept of converting time series into images for presses the internal correlation of the data, which enriches the infor­
diagnosis has been widely applied. A promising conversion method can mation. However, the GAF method can only encode a single-source
simplify complex time series into images and improve fault diagnosis signal into a grayscale image, and cannot make full use of the color
accuracy effectively. The GAF method enriches the information of the channels of the image. Moreover, it cannot be used for the fusion of dual-
image by adding correlation information while maintaining the original source signals.
information of the signal which improves the fault diagnosis accuracy
(Wang and Oates, 2015).
In two-dimensional space, the scalar product of two vectors is one of

3
G. Li et al. Expert Systems With Applications 237 (2024) 121521

represents the input of the neuron j at layer l; xl+1 i (j) represents the
output of the neuron j at layer l, which is the input at layer l + 1.
Each convolutional layer in ResNet is computed as a linear function.
Regardless of how many convolutional layers are added, it is equivalent
to a larger convolutional layer and the output is independent of the
number of layers in the neural network. Therefore, a nonlinear activa­
tion function is needed in each convolutional layer to enhance the
expressiveness and learning ability of the neural network. The rectified
linear unit (ReLU) function outputs the neuron directly when it is greater
than zero and makes the neuron zero when it is smaller than zero. The
most significant advantage of ReLU is its simplicity in computation, both
in forward and backward propagation of the derivative, which allows it
to be widely used in CNN. The expression for ReLU is as follow (He et al.,
2022)
{
f (x) = 0,x⩽0
x,x>0
Fig. 3. Residual functions.
To reduce network parameters and computational effort, preserve
features, and prevent overfitting, the pooling layer is employed to
2.2. ResNet reduce the size of the feature map by oversampling. There are two main
types of pooling, including average pooling and max pooling. The
Two-dimensional CNN (2D CNN) has been widely applied in the average pooling takes the average value of the features in the pooled
fields of fault diagnosis, machine vision, and medical care in recent years kernel as the output. The average pooling expression is (Zhang et al.,
due to its powerful feature extraction ability and translation invariance 2018)
(Gu et al. 2022; Vaillant et al., 1994; Chen et al., 2022). Models such as ( )
AlexNet (Krizhevsky et al., 2017), GoogleNet (Simonyan and Zisserman, l+1 1 ∑ s ∑ s
l
pu = 2 qu (ij)
2014), and ResNet (He et al., 2016) have been proposed successively. s i=1 j=1
Similar to traditional 2D CNN, ResNet has the same structure of input
layer, convolutional layers, pooling layers, and output layer. ResNet
requires a suitable loss function to handle specific problems, such as
Categorical Crossentropy (A. Rusiecki, 2019) for multi-classification Table 1
problems, Binary Crossentropy (Ruby et al., 2020) for binary classifi­ Structure parameters of ResNet model.
cation problems, and Mean-Squared Error (Wan, 1990) for regression Number Layer name Parameters Activation functions
problems. 1 Input 227 × 227 × 3
To solve the problem of gradient disappearance in the deep layer 2 Conv1 7 × 7, 64 stride 2 ReLU
network, ResNet employs residual functions as shown in Fig. 3. It not 3 BN1
only connects the layers sequentially in the order but also allows the 4 Max pooling1 3 × 3 stride 2
⎡ ⎤
5 Bottleneck1 1 × 1, 64 ReLU
shallow layer to skip the intermediate layer and directly connect to the ⎣ 3 × 3, 64 ⎦ × 3
deep layer, allowing the performance of the neural network to be un­ 1 × 1, 256
⎡ ⎤
affected even if the intermediate layer degrades. 6 Bottleneck2 1 × 1, 64 ReLU
⎣ 3 × 3, 64 ⎦ × 4
ResNet extracts input image features through convolutional layers.
1 × 1, 256
In a convolutional layer, the convolution kernel convolves the output 7 Bottleneck3
⎡
1 × 1, 256
⎤
ReLU
data from the previous layer to extract local features. The output of each ⎣ 3 × 3, 256 ⎦ × 6
layer is the result of the convolution of multiple input features, which is 1 × 1, 512
⎡ ⎤
8 Bottleneck4 1 × 1, 512 ReLU
described in the mathematical model as (Fu et al., 2017) ⎣ 3 × 3, 512 ⎦ × 3
1 × 1, 2048
xl+1 l l l
i (j) = ωi ∗ x (j) + bi 9 Global average pooling
10 Fully connected 1×1×9 Softmax
where ωli represents the weight of the convolutional kernel i at layer l;
11 Output
l
bi represents the bias of the convolution kernel i at layer l; xl+1 i (j)

Fig. 4. Structure of ResNet.

4
G. Li et al. Expert Systems With Applications 237 (2024) 121521

Fig. 5. The process of DS-GAF.

where q1u (ij) is the output of channel u, neuron ij in layer l; The size of fault diagnosis accuracy. Multi-source signal fusion methods provide a
the pooling kernel is s; The p1+1
u is the output of channel u in layer l + 1 of more comprehensive description of the working state of the machine
the pooling layer. and improve fault diagnosis accuracy effectively (Chao et al., 2022; Liu
Max pooling is output as the maximum value in the receptive field, et al., 2022; Shi et al., 2022; Yu et al.,2023). The original GAF method
with the expression (Zhang et al., 2016) expands the information by encoding the internal correlation of the time
{ l } series as an image. However, transforming the signal to polar co­
pl+1 = max qu (ij)
u
(v− 1)s+1<t<vs
ordinates only allows one GAF matrix to be created. The method encodes
(v− 1)s+1<t<vs directly into a greyscale image, which cannot fully utilize the color
channels in the image, limiting the application of the GAF method in
where q1u(ij) is the output of the channel u, neuron ij, in layer l; The size fault diagnosis. To address this problem, a dual-source Gramian angular
of the pooling kernel is s; The step size is v; p1+1
u is the output of the field method is proposed, which increases the number of input signals
channel u in layer l + 1 of the pooling layer. and enables signal fusion.
To further extract features a fully connected layer is applied. It is The procedure of the DS-GAF method for fusing signals is shown
usually connected to the final pooling layer and interconnects its output below.
features with all nodes of the fully connected layer.
The output layer is usually a softmax classifier, whose output is the (1) The DS-GAF method normalizes the time series and converts
probabilities of each label, and the maximum probability corresponding them into points in spherical coordinates. The polar angle θ and
to the label is recognized as the result. The model structure is shown in azimuthal angle φ of each point are calculated from the arccosine
Fig. 4. of the corresponding vibration signal amplitudes and strain signal
In this paper, a ResNet-50 model is employed, with the structural amplitudes, respectively.
parameters shown in Table 1. (2) Based on the position of each point in spherical coordinates, three
The ResNet for drilling pump fluid end fault diagnosis consists of a DS-GAF matrices are calculated from θ, φ, and θ-φ respectively,
convolutional layer, five residual blocks, a global average pooling layer, and are encoded the red, green, and blue color channels of the
a fully connected layer, and an output layer. The input image is image.
convolved by a convolution layer and residual blocks to obtain a feature (3) Combining three color channels as an image completes the DS-
map. The feature map is oversampled by pooling layers to reduce the GAF encoding for data fusion. The process of the DS-GAF
number of parameters. Then, all features are output as a vector of method is shown in Fig. 5.
dimensional width of 1 through a global average pooling layer. The fully
connected layer with softmax as the activation function obtains the The fusion of the vibration-strain signals for the method is shown in
probability values for each type of fault, and the fault with the highest Fig. 5. The figure shows that after converting the normalized vibration
probability is the diagnostic fault. and strain signals to spherical coordinates, each point in the coordinates
contains information about the time and the amplitude of the vibration
and strain signals. Three GAF DS-matrices are calculated to encode the
3. The proposed approach and experimental design
color channels based on the positional correlation of all points in the
coordinates. Three color channels compose the DC-GAF image, which
3.1. DS-GAF
contains information from both vibration and strain signals.
The DS-GAF method normalizes the raw vibration data X and strain
In fault diagnosis of drilling pump fluid end, a single-source signal
data Y to obtain the normalized time series. For the moment i, the
cannot describe the working state comprehensively, resulting in a low

Fig. 6. Three color channels for DS-GAF image.

5
G. Li et al. Expert Systems With Applications 237 (2024) 121521

Fig. 7. The process of the fault diagnosis method of drilling pump fluid end based on the DS-GAF and ResNet.

Fig. 8. Experiment procedure and experiment equipment.

Table 2
Classification of fault.
Label State of valve
Suction valve Discharge valve

SCDC Critical damage Critical damage

SCDM Critical damage Minor damage
SCDN Critical damage Normal
SMDC Minor damage Critical damage
SMDM Minor damage Minor damage
SMDN Minor damage Normal
SNDC Normal Critical damage
SNDM Normal Minor damage
SNDN Normal Normal

Fig. 9. Three states of the valve body. (a) Normal state, (b) Minor damage, and
(c) critical damage.
⎡ ⎤
cos(θ1 + φ1 ) cos(θ1 + φ2 ) ⋯ cos(θ1 + φN )
⎢ cos(θ2 + φ1 ) ⋯ cos(θ2 + φN ) ⎥
arccosine of the normalized amplitudes are assumed as polar angle θi DS − GAFB = ⎢
cos(θ2 + φ2 ) ⎥
⎣ ⎦
and the azimuthal angle φi, respectively. The radius ri is the ratio of that ⋮ ⋮ ⋱ ⋮
cos(θN + φ1 ) cos(θN + φ2 ) ⋯ cos(θN + φN )
moment i to the length N of the time series. Dual-source signals are
converted to spherical coordinates by Eq. (9). Three color channels are combined into an image to complete the DS-
⎧ GAF image, as shown in Fig. 6(d).
⎪
⎪
⎪
⎪
θi = arccos̃xi Fig. 6(d) shows a DS-GAF image with fused vibration-strain signals.
⎪
⎨
φi = arccos̃yi Each of the three color channels of the DS-GAF image is encoded with a
⎪
⎪ different signal. The red channel is encoded with the vibration signal, as
⎪ i
⎪
⎪
⎩ ri = shown in Fig. 6(a). The green channel is coded with the strain signal, as
N
shown in Fig. 6(b). The blue channel is encoded with the vibration-strain
Three color channels in the image, red, green, and blue, are encoded signal, as shown in Fig. 6(c).
separately through DS-GAF matrices. As shown in Fig. 6(a), the vibra­
tion signal is encoded as the red channel DS-GAFR matrix. 3.2. The process of the fault diagnosis method of drilling pump fluid end
⎡ ⎤
cos(θ1 + θ1 ) cos(θ1 + θ2 ) ⋯ cos(θ1 + θN )
⎢ cos(θ2 + θ1 ) cos(θ2 + θ2 ) ⋯ cos(θ2 + θN ) ⎥ The process of the fault diagnosis method of drilling pump fluid end
DS − GAFR = ⎢ ⎣
⎥
⎦ based on the DS-GAF and ResNet is shown in Fig. 7.
⋮ ⋮ ⋱ ⋮
cos(θN + θ1 ) cos(θN + θ2 ) ⋯ cos(θN + θN ) Acquisition of vibration signals and strain signals from the drilling
pump fluid end under multiple working conditions.
As shown in Fig. 6(b), the strain signal is encoded as the green Fusing vibration-strain signals into images by DS-GAF.
channel DS-GAFG matrix. Inputting DS-GAF fusion image datasets into ResNet to achieve fault
⎡ ⎤ diagnosis.
cos(φ1 + φ1 ) cos(φ1 + φ2 ) ⋯ cos(φ1 + φN )
⎢ cos(φ2 + φ1 ) cos(φ2 + φ2 ) ⋯ cos(φ2 + φN ) ⎥
⎢
DS − GAFG = ⎣ ⎥
⋮ ⋮ ⋱ ⋮ ⎦ 3.3. Data acquisition for experiments
cos(φN + φ1 ) cos(φN + φ2 ) ⋯ cos(φN + φN )
The experimental equipment and signal acquisition process is shown
As shown in Fig. 6(c), the vibration-strain signal is encoded as the
in Fig. 8. The five-cylinder drilling pump for the experiment is shown in
blue channel DS-GAFB matrix.
Fig. 8(a), with a rated power of 2400 HP, a maximum pump stroke of
188 SPM and an external dimension of 8076 × 3155 × 2910 mm. The

6
G. Li et al. Expert Systems With Applications 237 (2024) 121521

Fig. 10. Nine types of fault signals. (a) Vibration signals, and (b) strain signals.

Fig. 11. Images encoded by GAF and CWT. (a) GAF with strain, (b) GAF with vibration, (c) CWT with strain, and (d) CWT with vibration.

acceleration sensors S32D20 for acquiring vibration signals and the correlated with the experimental data, the starting point of the data is a
strain sensors CPR610-S01 for acquiring strain signals were installed on randomly selected point within 10% of the end of the previous piece of
each of the five suction valve boxes at the fluid end of the drilling pump, data. There are 2000 pieces of data for each type of fault under multiple
as shown in Fig. 8(b). A Siemens 1500 series PLC was deployed to collect working conditions, and the dataset contains 18,000 pieces of data in
and transmit the signals, as shown in Fig. 8(c). The sampling frequency total. Each piece of data contains 1000 sample points for vibration and
was set to 1 kHz to balance sampling accuracy and processing efficiency. strain signals, respectively.
The data was stored and pre-processed by the laptop, as shown in Fig. 8 After randomly distributing the data for each type of fault, 80% is
(d). selected as the training set, with 1600 pieces of data for each type of
There are three states of normal, Minor damage, and critical damage fault, for a total of 14,400 pieces. The remaining 20% is the test set, with
for both the suction valve and discharge valve respectively, as shown in 400 pieces of data for each type of fault, for a total of 3600 pieces.
Fig. 9.
A total of nine types of fault signals were collected for the experiment 4. Results and discussions
and their labels and corresponding faults are shown in Table 2.
In the experiments, the drilling pump was operated with pump stroke 4.1. Single-source signal fault diagnosis experiments
at 50 SPM, 90 SPM, 110 SPM, and pressure 10–40 MPa respectively
under different fault states. The vibration and strain signals from the To prove the superiority of the proposed DS-GAF method for drilling
drilling pump fluid end were obtained for multiple working conditions pump fluid end fault diagnosis, the vibration data and strain data are
and fault types. The 9 types of fault signals are shown in Fig. 10. encoded into images by the original GAF and CWT respectively.
Generated fault image size is 277 × 277 pixels. The fault image is shown
3.4. Fault data pre-processing on the drilling pump fluid end in Fig. 11.
To improve the convergence speed, the Adam optimizer (Kingma and
The experimental data are sliced into 1-second lengths. To extend the Ba, 2014) is used to iterate the neural network weights with a learning
sample size and to prevent the fault diagnosis results from being phase- rate of 0.0001. Adam algorithm is an extended form of stochastic

7
G. Li et al. Expert Systems With Applications 237 (2024) 121521

Fig. 12. Train histories of four datasets. (a) GAF with strain, (b) GAF with vibration, (c) CWT with strain, and (d) CWT with vibration.

Fig. 13. Confusion matrices of four datasets. (a) GAF with strain, (b) GAF with vibration, (c) CWT with strain, and (d) CWT with vibration.

gradient algorithm, which can update the neural network weights diagnostic model test sets are shown in Fig. 13.
adaptively based on the training data. The batch size in the model The horizontal coordinate of the confusion matrix represents the real
training is 16. The number of training iterations is 20. fault type and the vertical coordinate is the predicted fault type output
The model was trained on 64-bit Windows 10 with Intel Xeon Gold by the diagnostic model, with the expectation that the diagonal elements
5218 CPU, 128 GB memory, and Quadro RTX 5000 GPU. are close to 100%. The results of the above experiments show that the
Fig. 12 shows the train history of the four single-source signal fault accuracy of all test sets is lower than 90%. The lowest diagnostic accu­
diagnosis models. With iterative training of the neural network, it is racy is 80.06% for strain signals coded by GAF and the highest accuracy
expected that the higher accuracy and the lower loss. However, none of is 87.06% for vibration signals coded by CWT. For different methods,
the training accuracies is higher than 90% for all four single-source GAF encodes vibration signals with better accuracy than the CWT
signal image datasets. The highest is 87.10% for CWT with strain and method. However, it encodes stress signals with lower accuracy than
the lowest is 80.66% for GAF with strain. The lowest loss is 0.32 for CWT CWT. The accuracy of both methods is lower than 90% which cannot
with strain and the highest is 0.48 for GAF with strain. meet the accuracy requirements of field operation of the drilling pump.
The confusion matrices for the four single-source signal fault

8
G. Li et al. Expert Systems With Applications 237 (2024) 121521

4.2. Fault diagnosis experiments with DS-GAF

As shown in Fig. 14, nine fault types of images at the drilling pump
fluid end are generated by DS-GAF, with an image size of 227 × 227
pixels. Each DS-GAF image contains three color channels, red, green,
and blue, which are encoded by vibration, strain, and vibration-strain
DS-GAF matrices, respectively.
The DS-GAF dataset is input to the ResNet fault diagnosis model for
training. Train history is shown in Fig. 15. The Accuracy reaches 100%
at the 17th iteration, while the loss converges to 0.3. The accuracy of the
test set reaches 98.97%. The results show that the fault diagnosis model
based on DS-GAF is effective in diagnosing faults in the drilling pump
fluid end.
The confusion matrix for the test set is shown in Fig. 16. The accuracy
of the fault diagnosis model based on DS-GAF images is 98.97%. The
accuracy of the four faults in SMDC, SNDM, and SNDN reaches 100%.
The lowest fault diagnosis accuracy is 95.75% for SCDN. The most
serious misclassifications are SCDCs and SCDNs. SCDCs are misclassified
Fig. 14. Images encoded by DS-GAF. as SCDN in 1.75% of cases and misclassified as SNDCs and SNDNs in
0.25% of cases respectively. 1.75% of SCDCs are misclassified as SCDNs,
as well as 0.25% of SCDCs, are misclassified as SNDCs and SNDNs
respectively. 3.75% of SCDNs are misclassified as SCDCs and 0.5% of

Fig. 15. Train history of DS-GAF dataset.

Fig. 16. Confusion matrix of the test set of the DS-GAF dataset.

9
G. Li et al. Expert Systems With Applications 237 (2024) 121521

Fig. 17. Visualization of key layer features.

Table 3
Diagnosis results with different datasets.

Datasets Accuracy (%)

SCDC SCDM SCDN SMDC SMDM SMDN SNDC SNDM SNDN Mean

DS-GAF 97.75 99.75 95.75 100 98.25 99.75 99.50 100 100 98.97
GAF with strain 87.50 81 63 90.75 72 96.25 80 86.50 63.50 80.06
GAF with vibration 66.50 78.75 71 82 87.25 92.25 91.25 97.50 93 84.39
CWT with strain 77.25 80 76.50 96.75 95 99.75 92.25 90.25 75.75 87.06
CWT with vibration 74.25 69.25 60 72.5 93 87.5 84.75 96 97 81.58

SCDNs are misclassified as SNDCs. Similar to the single-source signal In the input layer, the different fault signals are mixed and cannot be
fault diagnosis model, SCDN shows the lowest accuracy in DS-GAF, GAF distinguished. As the network deepens, after several Bottlenecks, the
with strain, and CWT with vibration. Among all five fault diagnosis neural network gradually learns the features of each fault type and the
models, the average diagnostic accuracy of three fault types in critical signals of each fault type are gradually separated. In the output layer, the
damage to the suction valve is lower than Minor damage to the suction various fault signals are almost completely separated, with only the
valve and normal state to the suction valve. Among all the models, SCDC and SCDN partially overlapping and a small number of signals
critical damage to the suction valve shows the lowest average accuracy being classified into the wrong fault.
in CWT with vibration at 67.83%, which is increased to 97.75% after DS- The accuracies of each fault type are compared in Table 3 for the five
GAF fuses the vibration and strain data. dataset fault diagnosis models based on DS-GAF, GAF, and CWT,
As shown in Fig. 17, the output of the key layers of the test set is respectively. DS-GAF achieves a diagnostic accuracy of 98.85%, with
downscaled to two dimensions by the t-SNE method. Clustering scatter SMDC, SNDM, and SNDN achieving 100% diagnostic accuracy. The
plots are drawn to visualize the features of each layer. Each point in the diagnostic accuracies of the other four single-source signal datasets are
figure represents a piece of fault data. The color of the point represents lower than 90%. The DS-GAF fused vibration-strain dataset has higher
the fault type. The distance between points represents the variability of diagnostic accuracy than the GAF and CWT single-source signal datasets
the data, with the expectation that the same faults are clustered for all faults. The fault diagnosis accuracy of SCDN and SNDN in GAF
together. with strain is only 63% and 63.5%. After data fusion, the accuracy is

10
G. Li et al. Expert Systems With Applications 237 (2024) 121521

Fig. 18. Precision, recall, and F1 scores of the drilling pump fluid end fault diagnosis models. (a) Precision, (b) recall, and (c) F1 scores.

Fig. 21. Experimental setup for gearbox dataset by Southeast University (Shao
et al., 2018).

accuracy was improved significantly. The results show that the DS-GAF
method of fusing vibration-strain data improves the accuracy of fault
diagnosis significantly.
To further prove the superiority of the DS-GAF fault diagnosis model,
Fig. 19. Accuracy of different fault diagnosis methods. metrics including precision, recall, and Fscore were applied to evaluate
the five dataset fault diagnosis models based on DS-GAF, GAF, and CWT,
improved to 95.75% and 100%. The fault diagnosis accuracy of SCDC in respectively. When evaluating the output of a fault diagnosis model, TP
GAF with vibration is only 66.50%. After data fusion, the accuracy is (true positives) is applied to indicate the correct positive predictions. FN
improved to 97.75%. Comparing the single-source signal datasets of (false negatives) is applied to refer to the instances falsely labeled as
GAF and CWT, the average diagnostic accuracy of fault diagnosis based negative. TN (true negatives) is applied to represent the correctly
on DS-GAF is improved by 18.91%, 14.58%, 11.91%, and 17.39% identified negative cases. Based on these definitions, the three metrics
respectively. For the single-signal models, the diagnostic accuracy for all can be expressed as follow
three scenarios involving critical damage to the suction valve is rela­
tively low. However, after DC-GAF fused dual-source signals, the Precision = TP/(TP + FN)

Fig. 20. Precision, recall, and F1 scores of the drilling pump fluid end fault diagnosis approaches. (a) Precision, (b) recall, and (c) F1 scores.

11
G. Li et al. Expert Systems With Applications 237 (2024) 121521

Table 4 operation image data set and AlexNet (Li et al., 2023) has an accuracy of
Gearbox fault types description. 98.00%. The accuracies of CNN-LSTM, 1D-CNN, and RBF, based on
Type Description time-series data sets, are 88.33%, 83.22%, and 80.92%, respectively.
The lowest diagnostic accuracies are achieved by BP and SVM, based on
Health Health
Chipped Crack occurs in the gear feet time-series data sets, which are just higher than 60%, at 66.21% and
Miss Missing one of feet in the gear 60.08%, respectively. The comparison results show that the fault diag­
Root Crack occurs in the root of gear feet nosis method of the drilling pump fluid end based on the DS-GAF fused
Surface Wear occurs on the surface of the gear vibration-strain signal is more accurate than other methods. DC-GAF
fuses two dual-source signals and provides more fault information
than conventional approaches. Additionally, the deeper convolutional
Recall = TP/(TP + FN) layers and residual structure of ResNet, allow the model to capture
( ) /( ) complex features. Therefore, the fault diagnosis model based on DS-GAF
Fscore = 1 + β2 ⋅Precision⋅Recall β2 ⋅(Precision + Recall) and ResNet enables accurate diagnosis of drilling pump fluid end faults
The coefficient β in the Fscore is generally set at 1 which is called the under multiple working conditions.
F1 score. Precision measures the accuracy of model. Recall is a measure As shown in Fig. 20, the precision, recall, and F1 scores of ResNet,
of the ability of the model to identify all positive cases without missing Inception, AlexNet, VGG, CNN-LSTM, 1D-CNN, and RBF are analyzed.
any. The Fscore combines precision and recall into a single value. Higher All metrics of the DC-GAF ResNet model are higher than 95%. However,
values in all three metrics indicate better model performance. only the precisions of SCDM, SMDC, SMDM, and SNDM reach the
The three metrics precision, recall, and F1 score of the fault diag­ highest and all of them are 100%. The precisions of the remaining faults,
nostic models for the fluid end based on the five datasets are compared SCDC, SCDN, SMDN, SNDC, and SNDN, are lower than Inception, DC-
in Fig. 18. Every metric in the DS-GAF for each fault type reached the GAF Alexnet, VGG, or Alexnet, indicating that the DC-GAF ResNet
highest value, with all metrics above 95%. It is shown that the diagnosis model has a higher false alarm rate for these five types of faults.
model based on the DS-GAF method has the highest accuracy and the Moreover, the recall for SCND is lower than that of Inception and
lowest misclassification rate. The results prove the superiority of the DS- AlexNet, the recall for SMDN, and SNDC is lower than that of AlexNet,
GAF method. and the recall for SMDM is lower than that of VGG and CNN-LSTM,

4.3. Comparative analysis of different fault diagnosis methods

Table 5
Diagnosis results.
To further evaluate the effectiveness of the DS-GAF fault diagnosis
Method Accuracy of different methods
model proposed in this paper, it is compared with popular fault diag­
nosis methods, including Inception, AlexNet, VGG, CNN-LSTM, 1D- Transfer learning (Shao et al., 2018) 99.64%
CNN, radial basis function (RBF) neural network, support vector ma­ DS-GAF 99.90%
GAF with vibration 93.70%
chine (SVM), and BP neural network. The results are shown in Fig. 19. GAF with torque 99.70%
The accuracies of ResNet, Inception, AlexNet, and VGG based on DC- CWT with vibration 99.80%
GAF image data sets are higher than 90%, reaching 98.97%, 98.81%, CWT with torque 99.50%
98.22%, and 98.06% respectively. Fault diagnosis based on expansion

Fig. 22. Images encoded by DS-GAF, GAF, and CWT. (a) DS-GAF, (b) GAF with vibration, (c) GAF with torque, (d) CWT with vibration, and (e) CWT with torque.

12
G. Li et al. Expert Systems With Applications 237 (2024) 121521

Fig. 23. Precision, recall, and F1 scores of the gearbox fault diagnosis models. (a) Precision, (b) recall, and (c) F1 scores.

indicating that the DC-GAF ResNet model has lower sensitivity for these ResNet. The superiority of the proposed fault diagnosis method is veri­
four fault types. Additionally, the F1 Score for SCND, SMDN, SNDC, and fied. The main findings of this study are as follows.
SNDN is lower than that of AlexNet, and the F1 Score for SMDM is lower
than that of Inception, suggesting that the overall performance of DC- (1) GAF preserves time domain information and introduces correla­
GAF ResNet is slightly inferior to that of AlexNet or Inception for tion information when encoding time series signals into images,
these four fault types. However, the DC-GAF ResNet model still has the improving the information content of the dataset. The DS-GAF
highest overall performance. These identified limitations provide valu­ method increases the dual-source information fusion capability
able directions for future research in drilling pump fluid end fault based on the GAF method, which also provides a reference for the
diagnosis. signal-fusion diagnosis of other equipment.
(2) The fusion of vibration and strain signals provides a more accu­
5. Method adaptability investigation rate description of the working state of the drilling pump fluid
end. Compared to single vibration or strain signals, the fusion of
To evaluate the adaptability of fault diagnosis method proposed in vibration and strain signals can improve the accuracy of fault
this paper, a gearbox dataset was collected by Shao et al. (2018) from diagnosis at the drilling pump fluid end.
Southeast University. The experimental equipment is shown in Fig. 21. (3) Compared to traditional neural networks, our approach with the
As shown in Table 4, the gearbox dataset includes one healthy state deeper convolutional layers and residual structure of ResNet, the
and four fault states. The authors propose a transfer learning method ability of the model to capture complex features is enhanced, thus
with a fault diagnosis accuracy of 99.64%. surpassing some traditional neural networks. However, due to the
The motor torque and gearbox vibration are selected as input data complexity of DC-GAF and the deeper network structure of
and the data pre-processing method is the same as in section 3.4. Each ResNet, this study requires more computational resources and
fault type includes 1000 pieces of data, for a total of 5000 pieces of data. training time.
After randomly distributing the data for each fault type, 80% is selected (4) The drilling pump fluid end fault diagnosis method based on the
as the training set, with 800 pieces of data for each fault type, totaling DS-GAF fusion vibration-strain signal and ResNet can accurately
4000 pieces of data. 20% is selected as the testing set, with 200 pieces of diagnose the drilling pump fluid end fault, providing a new
data for each fault type, totaling 1000 pieces of data. Five image datasets approach to diagnosing drilling pump fluid end faults under
are encoded by DS-GAF, GAF, and CWT, as shown in Fig. 22. multiple working conditions.
Inputting all datasets into ResNet for fault diagnosis, the results are (5) The DS-GAF method proposed in this paper is effective in diag­
shown in Table 5. The highest fault diagnosis accuracy of 99.90% is nosing the drilling pump fluid end. However, further research is
achieved by DS-GAF in the fault diagnosis experiments on the gearbox needed to investigate its effectiveness in diagnosing faults in
dataset from Southeast University. DC-GAF improves fault diagnosis other components of the drilling pump. Subsequent studies will
accuracy by up to 6.20% over GAF and CWT. The results show that DS- be conducted to further improve the applicability of DC-GAF in
GAF improves fault diagnosis accuracy effectively. drilling pump failures.
A comparison of the precision, recall, and F1 scores of the five
models is shown in Fig. 23, which indicates that the fault diagnosis CRediT authorship contribution statement
model based on vibration signals performed the worst, with most met­
rics below 95%. However, after fusing the vibration signals with strain Gang Li: Conceptualization, Methodology, Funding acquisition.
signals by DS-GAF, the three metrics for each fault type achieved the Jiaxing Ao: Data curation, Investigation, Writing – original draft.
highest. The results demonstrate that the DS-GAF method improves the Jiayao Hu: Software, Writing – review & editing. Dalong Hu: Data
performance of the gearbox fault diagnosis model significantly, which curation. Yang Liu: Software. Zhiqiang Huang: Supervision, Writing –
also verifies the superiority of the DS-GAF method. review & editing.

Declaration of Competing Interest

6. Conclusion
The authors declare that they have no known competing financial
A fault diagnosis method based on DS-GAF and ResNet is proposed in
interests or personal relationships that could have appeared to influence
this paper to improve the accuracy of fault diagnosis of the drilling pump
the work reported in this paper.
fluid end. The vibration-strain data fusion image dataset is encoded by
the DS-GAF method, and the dataset is applied for fault diagnosis by

13
G. Li et al. Expert Systems With Applications 237 (2024) 121521

Data availability Li, G., Hu, J., Shan, D., Ao, J., Huang, B., & Huang, Z. (2023). A CNN model based on
innovative expansion operation improving the fault diagnosis accuracy of drilling
pump fluid end. Mechanical Systems and Signal Processing, 187, Article 109974.
Data will be made available on request. https://doi.org/10.1016/j.ymssp.2022.109974
Liang, P., Wang, W., Yuan, X., Liu, S., Zhang, L., & Cheng, Y. (2022). Intelligent fault
Acknowledgments diagnosis of rolling bearing based on wavelet transform and improved ResNet under
noisy labels and environment. Engineering Applications of Artificial Intelligence, 115,
Article 105269. https://doi.org/10.1016/j.engappai.2022.105269
This work was supported by Science and Technology Program of Liang, H., Cao, J., & Zhao, X. (2022). Average descent rate singular value decomposition
Sichuan Province (Grant No. 22GJHZ0284), and Nanchong City & and two-dimensional residual neural network for fault diagnosis of rotating
machinery. IEEE Transactions on Instrumentation and Measurement, 71, 1–16. https://
Southwest Petroleum University Science and Technology Strategic doi.org/10.1109/TIM.2022.3170973
Cooperation Special Fund (Grant No. SXHZ048). Liu, J., Wang, X., Wu, S., Wan, L., & Xie, F. (2023). Wind turbine fault detection based on
deep residual networks. Expert Systems with Applications, 213, Article 119102.
https://doi.org/10.1016/j.eswa.2022.119102
References Liu, Z., Xiao, W., Cui, J., & Mei, L. (2022). Application of an information fusion method
to the incipient fault diagnosis of the drilling permanent magnet synchronous motor.
Ayas, S., & Ayas, M. S. (2022). A novel bearing fault diagnosis method using deep Journal of Petroleum Science and Engineering, 219, Article 111124. https://doi.org/
residual learning network. Multimedia Tools and Applications, 81(16), 22407–22423. 10.1016/j.petrol.2022.111124
https://doi.org/10.1007/s11042-021-11617-1 Mitra, S., Mukhopadhyay, R., & Chattopadhyay, P. (2022). PSO driven designing of
Chao, Q., Gao, H., Tao, J., Wang, Y., Zhou, J., & Liu, C. (2022). Adaptive decision-level robust and computation efficient 1D-CNN architecture for transmission line fault
fusion strategy for the fault diagnosis of axial piston pumps using multiple channels detection. Expert Systems with Applications, 210, Article 118178. https://doi.org/
of vibration signals. Science China Technological Sciences, 65(2), 470–480. https:// 10.1016/j.eswa.2022.118178
doi.org/10.1007/s11431-021-1904-7 Pookkuttath, S., Gomez, B. F., Elara, M. R., & Thejus, P. (2023). An optical flow-based
Chen, S., Gamechi, Z. S., Dubost, F., van Tulder, G., & de Bruijne, M. (2022). An end-to- method for condition-based maintenance and operational safety in autonomous
end approach to segmentation in medical images with CNN and posterior-CRF. cleaning robots. Expert Systems with Applications, 222, Article 119802. https://doi.
Medical Image Analysis, 76, Article 102311. https://doi.org/10.1016/j. org/10.1016/j.eswa.2023.119802
media.2021.102311 Ruby, U., Theerthagiri, P., Jacob, J., & Yendapalli, V. (2020). Binary cross entropy with
Choudhary, A., Mian, T., & Fatima, S. (2021). Convolutional neural network based deep learning technique for image classification. International Journal of Advanced
bearing fault diagnosis of rotating machine using thermal images. Measurement, 176, Trends in Computer Science and Engineering, 9(10), 5393–5397. https://doi.org/
Article 109196. https://doi.org/10.1016/j.measurement.2021.109196 10.30534/ijatcse/2020/175942020
Cui, J., Zhong, Q., Zheng, S., Peng, L., & Wen, J. (2022). A lightweight model for bearing Rusiecki, A. (2019). Trimmed categorical cross-entropy for deep learning with label
fault diagnosis based on gramian angular field and coordinate attention. Machines, noise. Electronics Letters, 55(6), 319–320. https://doi.org/10.1049/el.2018.7980
10(4), 282. https://doi.org/10.3390/machines10040282 Shao, S., McAleer, S., Yan, R., & Baldi, P. (2018). Highly accurate machine fault diagnosis
Deng, S., Pei, J., Wang, Y., & Liu, B. (2017). Research on drilling mud pump fault using deep transfer learning. IEEE Transactions on Industrial Informatics, 15(4),
diagnosis based on fusion of acoustic emission and vibration technology. Insight-Non- 2446–2455. https://doi.org/10.1109/TII.2018.2864759
Destructive Testing and Condition Monitoring, 59(8), 415–423. https://doi.org/ Shao, X., & Kim, C. S. (2022). Unsupervised domain adaptive 1D-CNN for fault diagnosis
10.1784/insi.2017.59.8.415 of bearing. Sensors, 22(11), 4156. https://doi.org/10.3390/s22114156
Ding, C., Wang, Z., Ding, Q., & Yuan, Z. (2022). Convolutional neural network based on Shi, J. C., Ren, Y., Tang, H. S., & Xiang, J. W. (2022). Hydraulic directional valve fault
fast Fourier transform and gramian angle field for fault identification of HVDC diagnosis using a weighted adaptive fusion of multi-dimensional features of a multi-
transmission line. Sustainable Energy, Grids and Networks, 32, Article 100888. https:// sensor. Journal of Zhejiang University-SCIENCE A, 23(4), 257–271. https://doi.org/
doi.org/10.1016/j.segan.2022.100888 10.1631/jzus.A2100394
Dutta, N., Kaliannan, P., & Shanmugam, P. (2023). SVM Algorithm for Vibration Fault Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale
Diagnosis in Centrifugal Pump. Intelligent Automation & Soft Computing, 35(3), image recognition. arXiv preprint arXiv:1409.1556. doi: 10.48550/arXiv.1409.1556.
2997–3020. https://doi.org/10.32604/iasc.2023.028704 Tang, S., Zhu, Y., & Yuan, S. (2022). Intelligent fault diagnosis of hydraulic piston pump
Fu, Y., Zhang, Y., Gao, Y., Gao, H., Mao, T., Zhou, H., & Li, D. (2017). Machining based on deep learning and Bayesian optimization. ISA transactions, 129, 555–563.
vibration states monitoring based on image representation using convolutional https://doi.org/10.1016/j.isatra.2022.01.013
neural networks. Engineering Applications of Artificial Intelligence, 65, 240–251. Vaillant, R., Monrocq, C., & Le Cun, Y. (1994). Original approach for the localisation of
https://doi.org/10.1016/j.engappai.2017.07.024 objects in images. IEE Proceedings-Vision, Image and Signal Processing, 141(4),
Gu, J., Peng, Y., Lu, H., Chang, X., & Chen, G. (2022). A novel fault diagnosis method of 245–250. https://doi.org/10.1049/ip-vis:19941301
rotating machinery via VMD. CWT and improved CNN. Measurement, 200, Article Wan, E. A. (1990). Neural network classification: A Bayesian interpretation. IEEE
111635. https://doi.org/10.1016/j.measurement.2022.111635 Transactions on Neural Networks, 1(4), 303–305. https://doi.org/10.1109/72.80269
Guo, X., Lu, K., Cheng, Y., Zhao, W., Wu, H., Li, D., … Zhang, Y. (2022). Research on Wang, M., Wang, W., Zhang, X., & Iu, H. H. C. (2022). A new fault diagnosis of rolling
fault diagnosis method for hydraulic system of CFETR blanket transfer device based bearing based on Markov transition field and CNN. Entropy, 24(6), 751. https://doi.
on CNN-LSTM. Fusion Engineering and Design, 185, Article 113321. https://doi.org/ org/10.3390/e24060751
10.1016/j.fusengdes.2022.113321 Wang, Z., & Oates, T. (2015). Imaging time-series to improve classification and
Hao, D., & Gao, X. (2022). Unsupervised Fault Diagnosis of Sucker Rod Pump Using imputation. In Proceedings of the 24th International Conference on Artificial
Domain Adaptation with Generated Motor Power Curves. Mathematics, 10(8), 1224. Intelligence (pp. 3939-3945).
https://doi.org/10.3390/math10081224 Xiao, Q., Li, S., Zhou, L., & Shi, W. (2022). Improved variational mode decomposition
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. and CNN for intelligent rotating machinery fault diagnosis. Entropy, 24(7), 908.
In Proceedings of the IEEE conference on computer vision and pattern recognition https://doi.org/10.3390/e24070908
(pp. 770-778). doi: 10.1109/CVPR.2016.90. Yu, B., Chen, H., Jia, C., Zhou, H., Cong, L., Li, X., … Cong, X. (2023). Multi-modality
He, J., Li, L., & Xu, J. (2022). ReLU deep neural networks from the hierarchical basis multi-scale cardiovascular disease subtypes classification using Raman image and
perspective. Computers & Mathematics with Applications, 120, 105–114. https://doi. medical history. Expert Systems with Applications, 224, Article 119965. https://doi.
org/10.1016/j.camwa.2022.06.006 org/10.1016/j.eswa.2023.119965
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv Youcef Khodja, A., Guersi, N., Saadi, M. N., & Boutasseta, N. (2020). Rolling element
preprint arXiv:1412.6980, doi: 10.48550/arXiv.1412.6980. bearing fault diagnosis for rotating machinery using vibration spectrum imaging and
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. The International Journal of Advanced Manufacturing
convolutional neural networks. Communications of the ACM, 60(6), 84–90. https:// Technology, 106, 1737–1751. https://doi.org/10.1007/s00170-019-04726-7
doi.org/10.1145/3065386 Zhang, C., Xu, X., & Zhu, C. (2018). Image ordinal classification with deep multi-view
Lee, C. Y., Zhuo, G. L., & Le, T. A. (2022). A robust deep neural network for rolling learning. Electronics Letters, 54(22), 1280–1282. https://doi.org/10.1049/
element fault diagnosis under various operating and noisy conditions. Sensors, 22 el.2018.5101
(13), 4705. https://doi.org/10.3390/s22134705 Zhang, W., Qu, C., Ma, L., Guan, J., & Huang, R. (2016). Learning structure of
Li, C., Zhao, D., Mu, S., Zhang, W., Shi, N., & Li, L. (2019). Fault diagnosis for distillation stereoscopic image for no-reference quality assessment with convolutional neural
process based on CNN–DAE. Chinese Journal of Chemical Engineering, 27(3), 598–604. network. Pattern Recognition, 59, 176–187. https://doi.org/10.1016/j.
https://doi.org/10.1016/j.cjche.2018.12.021 patcog.2016.01.034

14