electronics

Article
An Improved Algorithm for Insulator and Defect Detection
Based on YOLOv4
Gujing Han 1,2, *, Qiwei Yuan 1,2 , Feng Zhao 3 , Ruijie Wang 1,2 , Liu Zhao 1,2 , Saidian Li 1,2 , Min He 4 , Shiqi Yang 4
and Liang Qin 4

1 School of Electronic and Electrical Engineering, Wuhan Textile University, Wuhan 430200, China
2 State Key Laboratory of New Textile Materials and Advanced Processing Technologies,
Wuhan Textile University, Wuhan 430200, China
3 State Grid Information & Telecommunication Group Co., Ltd., Beijing 102211, China
4 School of Electrical and Automation, Wuhan University, Wuhan 430072, China
* Correspondence: [email protected]

Abstract: To further improve the accuracy and speed of UAV inspection of transmission line insulator
defects, this paper proposes an insulator detection and defect identification algorithm based on
YOLOv4, which is called DSMH-YOLOv4. In the feature extraction network of the YOLOv4 model,
the improved algorithm improves the residual edges of the residual structure based on feature reuse
and designs the backbone network D-CSPDarknet53, which greatly reduces the number of parameters
and computation of the model. The SA-Net (Shuffle Attention Neural Networks) attention model is
embedded in the feature fusion network to strengthen the attention of target features and improve
the weight of the target. Multi-head output is added to the output layer to improve the ability of
the model to recognize the small target of insulator damage. The experimental results show that
the number of parameters of the improved algorithm model is only 25.98% of that of the original
model, and the mAP (mean Average Precision) of the insulator and defect is increased from 92.44% to
96.14%, which provides an effective way for the implementation of edge end algorithm deployment.

Keywords: UAV inspection; insulator defect; DSMH-YOLOv4; feature reuse; SA-Net; multi-head
Citation: Han, G.; Yuan, Q.; Zhao, F.;
Wang, R.; Zhao, L.; Li, S.; He, M.;
Yang, S.; Qin, L. An Improved
Algorithm for Insulator and Defect
Detection Based on YOLOv4.
1. Introduction
Electronics 2023, 12, 933. https:// As an important part of the overhead transmission line, insulators assume the role
doi.org/10.3390/electronics12040933 of mechanical support and electrical insulation of the line [1,2]. In addition, insulators
Academic Editors: Phivos Mylonas,
work under high voltage and high load for a long time and are often eroded by all kinds
Katia Lida Kermanidis and
of bad weather. Insulators are thus prone to damage, which seriously threatens the safety
Manolis Maragoudakis
and stability of transmission lines [3,4]. Therefore, the detection of insulators and defects
on high-voltage transmission lines has become an important task in power inspection. In
Received: 18 January 2023 recent years, due to its rapid rise, drone inspection technology has begun to gradually
Revised: 6 February 2023 replace manual inspections [5,6]. To achieve real-time high-precision detection of insulators
Accepted: 10 February 2023
and their defects by UAVs [7], the ability to deploy detection algorithms on UAVs at the
Published: 13 February 2023
edge is a necessary prerequisite [8,9]. At the same time, it is necessary to further improve
the recognition speed and accuracy of the algorithm for small targets such as insulator
defects in complex backgrounds, to improve the algorithm performance and increase the
Copyright: © 2023 by the authors.
detection efficiency [10,11].
Licensee MDPI, Basel, Switzerland.
With the continuous development of deep learning technology [12], methods related to
This article is an open access article the detection of insulators and their defects have been proposed one after another [13–15]. At
distributed under the terms and present, the commonly used deep learning algorithms are mainly divided into two categories:
conditions of the Creative Commons the first category comprises the regression-based one-stage algorithms, such as SSD (Sin-
Attribution (CC BY) license (https:// gle Shot MultiBox Detector), YOLO (You Only Look Once), YOLOv2, YOLOv3, and
creativecommons.org/licenses/by/ YOLOv4 [16–20]. The second category comprises the two-stage algorithm based on re-
4.0/). gion candidates, such as R-CNN (Region-Convolutional Neural Network), Fast R-CNN,

Electronics 2023, 12, 933. https://doi.org/10.3390/electronics12040933 https://www.mdpi.com/journal/electronics

Electronics 2023, 12, 933 2 of 16

Faster R- CNN, and Mask R-CNN [21–24]. The one-stage algorithm directly zones the
input image without generating candidate frames, providing faster detection than the
two-stage algorithm, but with slightly lower detection accuracy. However, after several
generations of development, the YOLO family of algorithms has the potential to com-
bine accuracy and speed and is one of the main algorithms in current research into target
detection applications.
YOLOv3, a classic algorithm in the first stage, is also quite good at recognizing small
targets while maintaining its speed advantage. Jia et al. [25] proposed a YOLOv3 improve-
ment algorithm for insulator detection, which can speed up the computation by improving
the residual structure in the backbone feature extraction network to a depth-separable
convolution; however, the accuracy is low. Liu et al. [26] replaced the YOLOv3 backbone
network with DenseNet, which effectively improved the feature extraction capability of
the images, but still used the original FPNs (Feature Pyramid Networks) network for the
detection of broken small targets, which was not sufficient for the recognition of small
targets. Zhu et al. [27] reduced the residual blocks in the YOLOv3 backbone network,
which reduced the network depth and the number of operations, but lost some of the
feature information, leading to a reduction in accuracy. Yang et al. [28] replaced the back-
bone feature extraction network of YOLOv3 with a lightweight Mobilenet network, which
sped up the computation. However, the extraction of features by downsampling does
not increase the number of channels sufficiently, the higher-level semantic information is
lost, and the accuracy is low. YOLOv4 is a successor to YOLOV3 in terms of speed and
accuracy, making it a valuable tool in the detection of insulators and defects. Yang et al. [29]
incorporated an attention mechanism into the YOLOv4 feature extraction network, which
enabled the model to better capture valid information, but with lower accuracy for target
detection in complex contexts. He et al. [30] introduced a feature fusion structure in the
YOLOv4 model to map shallow information to a feature pyramid and fuse deeper semantic
information, but the increased number of model parameters and slower detection speeds
led to poorer detection in UAV-embedded devices. Gao et al. [31] used migration learning
and super-resolution generation networks based on YOLOv4. The overall performance
of the model was improved and the network detection speed was accelerated, but the
detection accuracy of small targets for insulator defects was insufficient. Han et al. [32]
introduced the Self-Attention Mechanism and Channel Attention Mechanism modules in
Tiny-YOLOv4, which greatly reduced the complexity of the model and allowed portability
to embedded platforms; however, the model was slightly less accurate. Although the
above algorithms can better detect insulators, it is difficult to meet the requirements of
real-time recognition for small targets with broken insulators in complex contexts and these
targets cannot be detected in real time during UAV inspections. Most techniques increase
the size of the model while improving accuracy, making it difficult to perform embedded
applications on UAVs.
Based on the above analysis, this paper proposes an improved algorithm for insulator
and defect detection based on YOLOv4, called DSMH-YOLOv4, which further improves
the recognition accuracy of small targets with broken insulators in complex backgrounds
while increasing the insulator detection speed. The main improvements are as follows:
1. Replacing the original backbone feature extraction network with a lightweight back-
bone network, D-CSPDarknet53, through feature reuse, which greatly reduced the
number of parameters and computation of the model.
2. Embedding the SA-Net attention module [33] between the backbone network and
the feature fusion layer to improve the focus capability of the model for the complex
background where the detection target is located.
3. Adding multiple outputs to the prediction module to improve the detection accuracy
of the model for the small target of insulator defects.
Electronics 2023, 12, x FOR PEER REVIEW 3 of 16

Electronics 2023, 12, 933 3. Adding multiple outputs to the prediction module to improve the detection accuracy
3 of 16
of the model for the small target of insulator defects.

2. YOLOv4 Basic Structure

2. YOLOv4 Basic Structure
The YOLOv4 model consists of a feature extraction module, a feature enhancement
The YOLOv4 model consists of a feature extraction module, a feature enhancement
module,
module, and and aa detection
detectionmodule,
module,asasshown
shownininFigure
Figure1. 1.
In In
thethe
feature extraction
feature module,
extraction mod-
the
ule, the input image is first pre-processed to change the image size to 416 × 416 pre-
input image is first pre-processed to change the image size to 416 × 416 × 3. The × 3.
processed
The images are
pre-processed passed
images throughthrough
are passed the CSPDarknet53 backbone
the CSPDarknet53 network
backbone and three
network and
multi-scale featurefeature
three multi-scale maps containing different
maps containing dimensional
different information
dimensional are output:
information 13 × 13
are output:
× 1024,
13 × 1326 × ×1024,
26 × 512,
26 ×and
26 ×52512,
× 52 and
× 256.
52In×the
52 feature
× 256. enhancement
In the featuremodule, three different
enhancement module,
scales of feature
three different maps
scales ofneed to maps
feature be sampled
need toinbeboth directions
sampled and
in both stacked. and
directions Then, these
stacked.
feature maps are fully fused to obtain an optimized feature layer with more
Then, these feature maps are fully fused to obtain an optimized feature layer with more generalizabil-
ity. The detectionThe
generalizability. module usesmodule
detection the resulting optimized
uses the resultingfeature layer
optimized to identify
feature and
layer to local-
identify
ize targets at different scales and optimizes the output of the final prediction
and localize targets at different scales and optimizes the output of the final prediction through non-
maximum suppression.
through non-maximum suppression.

Figure 1. YOLOv4 structure diagram.

Figure 2 shows
shows the thestructure
structureofofthe theresidual
residualmodule,
module,The Theupper
upper layer
layerfeature
featureinforma-
infor-
tion Input
mation enters
Input thethe
enters residual
residualmodule
module andandis directly extracted
is directly extracted by by
thethe
convolution
convolution units of
units
thethe
of twotwopaths to to
paths obtain
obtain thethe
feature
featurematrices
matrices X and
X andY. Y.
X will
X willcontinue
continue to to
enter thethe
enter residual
resid-
unit.unit.
ual After extracting
After features
extracting by two
features by convolutions,
two convolutions, X stacks itself itself
X stacks to form the residual
to form struc-
the residual
ture. In the figure, n indicates the number of stacks of residual units,
structure. In the figure, n indicates the number of stacks of residual units, and the function and the function of
the feature extraction module is mainly achieved by five
of the feature extraction module is mainly achieved by five residual modules,residual modules, Resblock_body
(n = 1, 2, 8, 8, 4).(nY= 1,
Resblock_body is 2,
transmitted
8, 8, 4). Y isto the bottomtoof
transmitted thethe module
bottom viamodule
of the a residual via path with
a residual
a large
path span.
with It is then
a large span.stacked
It is thenandstacked
convolved and with the residual
convolved with the unitresidual
output feature Z to
unit output
feature Z to output the extracted feature Output. This operation allows for maximum the
output the extracted feature Output. This operation allows for maximum retention of re-
originalofinformation.
tention the original As shown in As
information. Figureshown1, the SPP (Spatial
in Figure Pyramid
1, the SPP Pooling)
(Spatial Pyramidnetwork
Pool-
after network
ing) the 5 residual modules
after the then processes
5 residual modules then the final outputthe
processes feature
final layer
output having a size
feature of
layer
13 × 13 × 1024. This feature layer is first convolved to further extract
having a size of 13 × 13 × 1024. This feature layer is first convolved to further extract fea- features. Then, it
is maximally pooled through 1 × 1, 5 × 5, 9 × 9, and 13 × 13 maximum
tures. Then, it is maximally pooled through 1 × 1, 5 × 5, 9 × 9, and 13 × 13 maximum pooling pooling kernels.
Finally, the
kernels. results
Finally, theofresults
each pooling
of each are stacked
pooling areand output.
stacked and output.
The acquisition of deep image features through the series of residual structures de-
scribed above leads to a large number of parameters in the backbone network, which adds
significantly to the complexity of the model. Thus, lightweight improvements are necessary
to suit the requirements of edge-end deployments, while maintaining the accuracy of the
model detection.
Electronics 2023,
Electronics 12,12,
2023, x FOR
933 PEER REVIEW 4 of4 of
1616

Figure 2. Residual module structure diagram. Conv: Convolution with kernel size 3 × 3, Strides = 2;
BN: Batch Normalization; Relu: Rectified Linear Unit.

The acquisition of deep image features through the series of residual structures de-
scribed above leads to a large number of parameters in the backbone network, which adds
significantly to the complexity of the model. Thus, lightweight improvements are neces-
sary to suit the requirements of edge-end deployments, while maintaining the accuracy
of the model detection.

3.Figure
Improved
Figure 2. 2. YOLOV4
Residual
Residual module Model
module
structureNetwork
structure diagram.
diagram. Structure
Conv:
Conv: Convolutionwith
Convolution withkernel size3 3× ×
kernelsize 3, 3, Strides
Strides = 2;
= 2;
BN:
BN: Batch
Batch Normalization;
Normalization; Relu:
Relu: Rectified
Rectified Linear
Linear Unit.
Unit.
3.1. Subsection Feature Extraction Module Lightweighting Improvement
Convolutional
3. Improved YOLOV4 neural networks
Model Network can lead to degradation problems if the performance
Structure
The acquisition of deep image features through the series of residual structures de-
of the network
3.1. Subsection is enhanced
Feature only by increasing the depth of the network. That is, the gra-
scribed above leads to a Extraction
large number Module Lightweighting
of parameters in theImprovement
backbone network, which adds
dient disappears as the network deepens to a certain point, resulting in a loss of accuracy.
Convolutional
significantly neural networks
to the complexity of the can leadThus,
model. to degradation
lightweight problems if the performance
improvements are neces-of
However, residual neural networks can deepen the information fusion between different
theto
sary network
suit theisrequirements
enhanced onlyofby increasing
edge-end the depth ofwhile
deployments, the network.
maintainingThat is,
thethe gradient
accuracy
feature layers by establishing short-circuit links [34]. This allows a deeper model to be
ofdisappears as the network deepens to a certain point, resulting in a loss of accuracy.
the model detection.
built that still has better recognition results.
However, residual neural networks can deepen the information fusion between different
The DenseNet network [35] is a modified residual structure. It is possible to create a
3. feature
Improvedlayers by establishing
YOLOV4 short-circuit
Model Network links [34]. This allows a deeper model to be built
Structure
tight fully connected structure between the upper and lower feature layers, thus effec-
that
3.1. still has
Subsection better recognition results.
tively linkingFeature Extraction
the features Module
between theLightweighting
different feature Improvement
layers. It can also be used to re-
The DenseNet network [35] is a modified residual structure. It is possible to create a
duce
tight the number
Convolutional
fully connected of structure
neuralmodel parameters
networks
between theand
can lead computational
to degradation
upper and lowerproblems effortlayers,
feature bythe
if communicating
performance
thus effectively
ofthrough
the network
feature is enhanced
reuse in only
the by
process increasing
of the
establishing depth
tight of
linking the features between the different feature layers. It can also be used the network.
connections. TheThat is, to
structurethereduce
gra-
of its
dient disappears
the number as the network deepens to a certain point, resulting in a loss of accuracy.
T0 outputs T1
feature reuseofDenseBlock
model parametersis shown andincomputational
Figure 3. Theeffort inputbyfeature
communicating through
However, residual
feature reuse neural
in the networks
process can deepen
of establishing theconnections.
tight information The fusion between
structure of different
its feature
feature
reuse layers
through by establishing
the transfer
DenseBlock isfunction HFigure
shown short-circuit
in links
0 ; at the3.same
The time,
input Tfeature
[34]. This0 isallows a deeper
transmitted
T0 outputs Tmodel
directly to be
from
1 through the
the
built that still
transfer has better
function H recognition results.
T0 is transmitted directly from the branch. T0 and T1
branch. T0 and 0 ; at the samethe
Tnetwork time,
function ItH
The DenseNet 1 constitute[35] is a input of the
modified transfer
residual structure. is .
1possible to create a
constitute the input of the transfer function H1 .
tight fully connected structure between the upper and lower feature layers, thus effec-
tively linking the features between the different feature layers. It can also be used to re-
duce the number of model parameters and computational effort by communicating
through feature reuse in the process of establishing tight connections. The structure of its
feature reuse DenseBlock is shown in Figure 3. The input feature T0 outputs T1
through the transfer function H 0 ; at the same time, T0 is transmitted directly from the

branch. T0 and T1 constitute the input of the transfer function H1 .

Figure DenseBlockstructure
Figure3.3.DenseBlock structurediagram.
diagram.

The calculation process is shown in Equation (1):

TL = HL−1 ( T0 , T1 , · · · , TL−1 ) (1)

In Equation (1), TL denotes the input at layer L.

Compared with the original residual structure, DenseNet’s residual edges directly
connect the upper and lower features, using the original upper features combined with the
lower features. This reduces the process of calculating the residual edges in the original
Figure 3. DenseBlock
residual structure structure diagram.
and reduces the model volume. It reduces computation time by directly
 TL = H L−（
1 T0 ,T1 , ,TL −1） (1)

In Equation (1), TL denotes the input at layer L.

Compared with the original residual structure, DenseNet’s residual edges directly
Electronics 2023, 12, 933 connect the upper and lower features, using the original upper features combined5with of 16
the lower features. This reduces the process of calculating the residual edges in the origi-
nal residual structure and reduces the model volume. It reduces computation time by di-
rectly upper-level
using using upper-level
feature feature parameters
parameters and alsoand alsofor
allows allows forutilization
higher higher utilization of shal-
of shallow-level
low-level information. It also prevents the problem of local information loss as
information. It also prevents the problem of local information loss as the network deepens, the net-
work deepens, speeds up convergence, and improves
speeds up convergence, and improves detection efficiency. detection efficiency.
Therefore,this
Therefore, thispaper
paperuses
usesthe
the idea
idea of of DenseNet
DenseNet using
using D-CSPDraknet53
D-CSPDraknet53 as backbone
as the the back-
bone network of the model, as shown
network of the model, as shown in Figure 4.in Figure 4.

Figure
Figure 4.4. D-CSPDraknet53
D-CSPDraknet53structure diagram.33××3 MaxPool2D:
structurediagram. 3 MaxPool2D: Max-Pooling
Max-Pooling with kernel
with 3 ×33,×
sizesize
kernel
2; 7 × 7 Conv: Convolution with kernel size 7 × 7, Strides = 2; 7 ×
3, Strides = 2; 7 × 7 Conv: Convolution with kernel size 7 × 7, Strides = 2; × 7 Conv: Convolution
Strides = 7 Conv: Convolution with
kernel size 7 size
with kernel × 1 Conv:
× 7; 71 ××7;1 1Conv: Convolution with kernel
Convolution size 1 ×size
with kernel 2 × 2 AvgPool2D:
1; 21×× 21;AvgPool2D: Avg-PoolingAvg-
Pooling
with with
kernel size 2 × 2,
kernel size 2 × 2,=Strides
Strides 2. = 2.

The
The five
five original
original residual
residual modules,
modules, Resblock_body, are replaced replaced byby aa maximum
maximum
pooling
pooling and four DenseBlocks
and four DenseBlocks (m (m ==6,6,12,
12,24,
24,16),
16),with
withmmbeing
being the
the number
number of of stacks
stacks of
of DenseBlocks.
DenseBlocks. ThisThis design
design approach
approach makes
makes the the connection
connection between
between the layers
the layers of net-
of the the
network tighter,
work tighter, greatly
greatly simplifies
simplifies themodel
the modelcomplexity,
complexity,reduces
reducesthethenumber
number ofof operations,
operations,
and improves the model detection speed. This type of connection
and improves the model detection speed. This type of connection mitigates gradient mitigates gradient
dis-
disappearance during network training. However, at the same time, the
appearance during network training. However, at the same time, the feature extraction feature extraction
effect of the
effect of themodel
modelisisreduced
reducedandandthethe accuracy
accuracy is slightly
is slightly degraded.
degraded. Thus, Thus, attention
attention mech-
mechanism can be embedded to compensate and simultaneously enhance
anism can be embedded to compensate and simultaneously enhance the model’s detection the model’s
detection
accuracy in accuracy
complex in background
complex background
situations.situations.
3.2. Improvements to Enhance Feature Focus Based on Attention Mechanisms
3.2. Improvements to Enhance Feature Focus Based on Attention Mechanisms
The attention mechanism takes a reference from the human behavior of selectively
The attention mechanism takes a reference from the human behavior of selectively
focusing on the important parts of the received information and constructs a model that
focusing on the important parts of the received information and constructs a model that
can redistribute the weight of target information from the irrelevant information in the
can redistribute the weight of target information from the irrelevant information in the
information received by the network. The spatial attention mechanism is primarily de-
information received by the network. The spatial attention mechanism is primarily de-
signed to capture the correlation between pixel points in the input features. The channel
signed tomechanism
attention capture theaims
correlation between
to enhance pixel points
the feature in the
channels andinput features.
amplify The weights.
the target channel
attention mechanism aims to enhance the feature channels and amplify the target
Using them in combination results in better performance, but also inevitably increases weights.
Using
the them in combination
computational effort andresults
numberin better performance,
of parameters of thebut also inevitably
model. This paper increases the
introduces
the SA-Net attention model using the idea of grouped convolution. SA-Net is not only a
lightweight feature attention module but also allows feature attention in different dimen-
sions. As shown in Figure 5, SA-Net receives the input features and splits the features into G
sub-features in the channel dimension. The principle is shown in Equation (2). The feature
weights are then updated in a parallel processing manner for each group of sub-features
in multiple dimensions. To minimize the number of parameters in the attention model,
Electronics 2023, 12, 933 6 of 16

SA-Net further dichotomizes the groups of sub-features by the number of channels. The
process is shown in Equation (3):

X = [ X1 , · · · , XG ], Xk ∈ RC/G× H ×W (2)
Electronics 2023, 12, x FOR PEER REVIEW 7
Xk = [ Xk1 , Xk2 ], Xk1 , Xk2 ∈ RC/2G× H ×W (3)

Figure 5. SA-Net structure diagram.

Figure 5. SA-Net structure diagram.
In Equations (2) and (3), X is the input feature and X ∈ RC× H ×W , C is the number of
3.3.HImprove
input channels, Detection
is the input Module
height, W is for
theSmall
inputTarget
width,Recognition
Xk is the kth sub-feature.
Whilefeatures
The dichotomized considering the model’s
are rescaled light weight
for channel and targeting
and spatial attentionofweights
complex sobackgrou
that the modelenhancing
focuses onthechannel
algorithm’s
and ability
locationtoinformation
recognize small targets
relevant is also
to the key to
target. Theevaluating
algorithm’s
principle of the channel performance.
attention module is shown in Equations (4) and (5), and the
As can
principle of the spatial be seenmodule
attention from Figure 6, the
is shown insulator (6):
in Equation defect targets occupy too small an
within the insulator as a whole. A large amount of shallow information is lost when
H W
features are used for prediction, 1 resulting in deep features accounting for more g
× W i∑ ∑ XThis
s = Fgp ( Xk1 ) = k1 (i, j ) (4)
information and less detailedHinformation. =1 j =1
is not conducive to the identificatio
small targets for insulator defects.
0
Xk1 = σ( F (s)) · Xk1 = σ(W1 s + b1 ) · Xk1 (5)
0
Xk2 = σ(W2 · GN ( Xk2 ) + b2 ) · Xk2 (6)
In Equations (4)–(6), Fgp is the global average pooling operation, σ is the sigmoid
function, GN is the group normalization operation, W1 , W2 , b1 , b2 are the parameters and
W1 , W2 , b1 , b2 ∈ RC/2G×1×1 .
The two-dimensional branches are then connected to output a new feature layer after
feature attention, as shown in Formula (7):

Xk0 = [ Xk1
0 0
, Xk2 ] ∈ RC/G× H ×W (7)

Finally, all updated sub-features are re-aggregated. However, there is no information

communication between the feature (a) layers. They are independent of each other
(b)and the
feature updates in the channel dimension and the spatial dimension cannot be fused. Thus,
it is also necessary to describe the joint relationship of features in space and channels in
terms of channel interaction before output. This results in information fusion between
different sub-features at the pixel level and makes the SA-Net module more efficient.

3.3. Improve Detection Module for Small Target Recognition

While considering the model’s light weight and targeting of complex backgrounds,
enhancing the algorithm’s ability to recognize small targets is also key to evaluating the
algorithm’s performance.
As can be seen from Figure 6, the insulator defect targets occupy too small an area
within the insulator as a whole. A large amount of shallow information is lost when deep
features are used for prediction, resulting in deep features accounting for more global
(c) (d)
 3.3. Improve Detection Module for Small Target Recognition
While considering the model’s light weight and targeting of complex backgrounds,
enhancing the algorithm’s ability to recognize small targets is also key to evaluating the
algorithm’s performance.
Electronics 2023, 12, 933 As can be seen from Figure 6, the insulator defect targets occupy too small an 7 ofarea
16
within the insulator as a whole. A large amount of shallow information is lost when deep
features are used for prediction, resulting in deep features accounting for more global
information
informationandandless
lessdetailed
detailedinformation.
information. This
This is
is not conducive
conduciveto
tothe
theidentification
identificationofof
small
small targets
targets forinsulator
for insulatordefects.
defects.

(a) (b)

Electronics 2023, 12, x FOR PEER REVIEW 8 of 16

(c) (d)

(e) (f)
Figure 6. 6.
Figure (a–f) Insulator
(a–f) Insulatordefect
defectpictures.
pictures.

TheThe principle
principle ofof convolution
convolution operation
operation is is shown
shown inin Figure
Figure 7; 7; a 5a×55×feature
when
when 5 feature
map
is map is convolved
convolved once, aonce,
newa3new 3 × 3 feature
× 3 feature map is map is obtained.
obtained. The S26-pixel
The S26-pixel pointpoint in this
in this new
feature map will contain the feature information from the 9 pixel points from S1 toS9.
new feature map will contain the feature information from the 9 pixel points from S1 to S9.
After
After anotherconvolution,
another convolution,the
thepixel
pixel S27
S27 will
will contain
contain the
the feature
featureinformation
informationofofthe 2525
the pixel
pixel
points from S1 to S25. Each detail of information in the 5 × 5 feature layer is given less
points from S1 to S25. Each detail of information in the 5 × 5 feature layer is given less
weight in S27, which has higher overall information content. The network is deepened
weight in S27, which has higher overall information content. The network is deepened
and the shallow detail information is compressed so that small target features cannot be
and the shallow
easily detail information is compressed so that small target features cannot be
identified.
easily identified.
 is convolved once, a new 3 × 3 feature map is obtained. The S26-pixel point in this new
feature map will contain the feature information from the 9 pixel points from S1 to S9.
After another convolution, the pixel S27 will contain the feature information of the 25 pixel
points from S1 to S25. Each detail of information in the 5 × 5 feature layer is given less
weight in S27, which has higher overall information content. The network is deepened
Electronics 2023, 12, 933 8 of 16
and the shallow detail information is compressed so that small target features cannot be
easily identified.

Figure 7. Convolution operation principle.

Therefore, in in this
this paper,
paper, we
weaddaddaashallow
shallowfeature
featurelayer
layerwith
with a higher
a higher resolution
resolution of
of
104104 × 104
× 104 to the
to the original
original three
three feature
feature detection
detection layers
layers of theofYOLOv4
the YOLOv4 network.
network. This
This oper-
operation facilitates
ation facilitates the identification
the identification and localization
and localization of smallofbroken
small targets
brokenbytargets by the
the detection
detection
module and module and improves
improves the model’stheability
model’s ability tosmall
to identify identify small
broken broken targets.
targets.
3.4. DSMH-YOLOv4 Model
3.4. DSMH-YOLOv4 Model
The improved algorithm proposed in this paper is manifested in the following three
The improved algorithm proposed in this paper is manifested in the following three
aspects: firstly, the original YOLOv4 backbone feature extraction network is replaced by D-
aspects: firstly, the original YOLOv4 backbone feature extraction network is replaced by
CSPDraknet53; secondly, the SA-Net attention mechanism is added between the backbone
D-CSPDraknet53; secondly, the SA-Net attention mechanism is added between the back-
feature extraction network and the feature fusion layer; thirdly, a shallow feature layer is
bone feature extraction network and the feature fusion layer; thirdly, a shallow feature
Electronics 2023, 12, x FOR PEER REVIEW 9 of 16
added to the feature detection layer to obtain more small target information. The structure
layer is added to the feature detection layer to obtain more small target information. The
diagram of the DSMH-YOLOv4 model is shown in Figure 8.
structure diagram of the DSMH-YOLOv4 model is shown in Figure 8.
The input image is extracted by the backbone network to obtain four effective feature
layers of different sizes. In addition, the SA-Net rescales the initial effective feature layers
from the miscellaneous ones to amplify the target weights and update the information
interaction in the feature layers. PANet performs bi-directional sampling fusion using a
feature layer with the target weights amplified by an attention mechanism, combining the
information in the different scale feature layers to provide a database for improving the
accuracy of the detection module.

Figure
Figure 8.
8. DSMH-YOLOv4
DSMH-YOLOv4 structure
structure diagram.

The input image

4. Experimental is extracted
Results by the backbone network to obtain four effective feature
and Analysis
layers of different sizes.
4.1. Experimental Environment In addition, the SA-Net rescales the initial effective feature layers
from the miscellaneous ones to amplify the target weights and update the information
This study used a deep learning framework based on the PyTorch 1.6 environment,
interaction in the feature layers. PANet performs bi-directional sampling fusion using a
with Ubuntu
feature 18.08,the
layer with Python
target3.8.0, andamplified
weights CUDA = 11.2,
by anwhere the training
attention mechanism,graphics card con-
combining the
figuration was an RTX A6000/48G graphics card.
information in the different scale feature layers to provide a database for improving the
As this
accuracy experimental
of the dataset is partly from a publicly available dataset and partly
detection module.
collected in the field, the data were amplified and screened for labeling. The total number
of
4. datasets was 1588,
Experimental of which
Results the ratio of the training set, validation set, and test set was
and Analysis
8:1:1. This resulted
4.1. Experimental in a training set of 1272, a validation set of 158, and a test set of 158.
Environment
The number of positive
This study and negative
used a deep learningsamples in the
framework dataset
based wasPyTorch
on the the number of insulators
1.6 environment,
and insulator breakage samples. The numbers were 2908 and 715,
with Ubuntu 18.08, Python 3.8.0, and CUDA = 11.2, where the training graphics respectively, withcard
an
overall ratio ofwas
configuration around 4:1. A6000/48G graphics card.
an RTX

4.2. Experimental Process

During the training process, freeze training and thaw training were performed for
the disclosed target detection model by loading the pre-trained model using the idea of
migration learning. The frozen layer was trained with 50 epoch rounds and 300 rounds of
Electronics 2023, 12, 933 9 of 16

As this experimental dataset is partly from a publicly available dataset and partly
collected in the field, the data were amplified and screened for labeling. The total number
of datasets was 1588, of which the ratio of the training set, validation set, and test set was
8:1:1. This resulted in a training set of 1272, a validation set of 158, and a test set of 158. The
number of positive and negative samples in the dataset was the number of insulators and
insulator breakage samples. The numbers were 2908 and 715, respectively, with an overall
ratio of around 4:1.

4.2. Experimental Process

During the training process, freeze training and thaw training were performed for
the disclosed target detection model by loading the pre-trained model using the idea of
migration learning. The frozen layer was trained with 50 epoch rounds and 300 rounds of
thawing training. For the improved model, retraining was used, again set for 300 rounds.
The initial learning rate of the model was 0.001, the batch size was set to 16, the momentum
was 0.9, and the weight decay was 0.0005.
The loss curve obtained from the training is shown in Figure 9, where the horizontal
coordinate represents the number of training rounds and the vertical coordinate is the
loss change value during the training process. As shown in Figure 9, the loss values are
greatly reduced at the beginning of the training process, indicating that the learning rate
is appropriate. The curve levels off when the iteration reaches 200 epochs, indicating that
the model had converged. The training loss and the validation loss of DMSH-YOLOv4
Electronics 2023, 12, x FOR PEER REVIEW 10 of 16
are lower than those of YOLOv4, respectively, and DMSH-YOLOv4 converges faster than
YOLOv4. It is concluded that our model has lower overfitting and higher accuracy.

Figure
Figure9.9.Loss
Losscurve
curveduring
duringtraining.
training.

4.3.Experimental
4.3. ExperimentalEvaluation
EvaluationIndicators
Indicators
To compare
To compare thetheperformance
performanceofofdifferent algorithms,
different thethe
algorithms, performance of the
performance ofalgorithms
the algo-
was evaluated using average precision (AP), mean average precision (mAP),
rithms was evaluated using average precision (AP), mean average precision (mAP), pa- parameters,
model size,
rameters, FLOPs,
model size,and detection
FLOPs, speed. speed.
and detection
The average precision is the sum of the area of the curve enclosed by precision (P)
The average precision is the sum of the area of the curve enclosed by precision (P)
and recall (R). The P-R graph is shown in Figure 10. The mAP is the average of the APs
and recall (R). The P-R graph is shown in Figure 10. The mAP is the average of the APs of
of the targets to be classified. Accuracy refers to the probability that a positive sample is
the targets to be classified. Accuracy refers to the probability that a positive sample is
predicted out of all the samples predicted to be positive, and recall refers to the probability
predicted out of all the samples predicted to be positive, and recall refers to the probability
that a positive sample is predicted out of the positive samples. The number of parameters
that a positive sample is predicted out of the positive samples. The number of parameters
affects the size of the model and the amount of computation affects the speed of the model.
affects the size of the model and the amount of computation affects the speed of the model.
FLOPs refer to the number of floating point operations, i.e., the amount of computation,
FLOPs refer to the number of floating point operations, i.e., the amount of computation,
and are often used to measure the complexity of the model. FPS is the number of images
and are often used to measure the complexity of the model. FPS is the number of images
per second detected by the model and is often used to measure the detection speed of the
model. Accuracy, recall, AP value, and mAP are calculated as shown in Equations (8)–(11):

TP
P= (8)
 Electronics 2023, 12, 933 10 of 16

per second detected by the model and is often used to measure the detection speed of the
model. Accuracy, recall, AP value, and mAP are calculated as shown in Equations (8)–(11):

TP
P= (8)
TP + FP
TP
R= (9)
TP + FN
ZR
AP = P(r )dr (10)
0
Electronics 2023, 12, x FOR PEER REVIEW APinsulator + APdefect 11 of 16
mAP = (11)
2

(a) (b)
Figure
Figure10.
10.(a)(a)Insulator
InsulatorP-R
P-Rcurve
curvediagram;
diagram;(b)
(b)defect
defectP-R
P-Rcurve
curvediagram.
diagram.

In Equations
4.4. Comparison TP is the number of samples that the model considers to be
(8) and (9), Results
of Experimental
Table 1 shows the test resultsFP
positive and are indeed positive. of is the number
comparison of samples that
experiments the model
between considersinto
the algorithm
be positive but which are actually negative. FN is the number of samples
this paper and the current mainstream algorithms Fast R-CNN, SSD, YOLOv3, YOLOv4,that the model
considers to be negative but which are actually positive.
YOLOv5, and YOLOX, with the same dataset and experimental parameters in this paper.
4.4. Comparison of Experimental Results
Table 1. Comparison of evaluation indicators of mainstream detection algorithms.
Table 1 shows the test results of comparison experiments between the algorithm in
Model Insulators AP (%)paper
this Defects
andAPthe(%) mAPmainstream
current (%) Parameters ModelFast
algorithms SizeR-CNN,
(MB) FLOPs
SSD, (G) SpeedYOLOv4,
YOLOv3, (FPS)
Faster R-CNN 94.36 YOLOv5, and 75.74 85.06the same
YOLOX, with 137,098,724 522.99
dataset and experimental 370.406
parameters 9.68paper.
in this
SSD 90.18 84.95 87.57 26,285,486 100.27 62.798 35.62
YOLOv3 92.15 Table 1. Comparison
91.27 of evaluation
91.71 indicators
61,949,149of mainstream detection algorithms.
236.32 66.096 23.58
YOLOv4 93.32 91.56 92.44 64,363,101 245.53 60.334 19.97
Insulators AP Defects AP Model Size
Model
YOLOv5s 93.30 mAP (%) Parameters FLOPs (G) Speed
58.18(FPS)
(%) (%)90.63 91.96 7,276,605 27.76
(MB) 17.060
YOLOXs
Faster R-CNN 96.35
94.36 91.88
75.74 94.11
85.06 8,968,255
137,098,724 34.21
522.99 26.806
370.406 46.18
9.68
DSMH-YOLOv4
SSD 95.75
90.18 96.54
84.95 96.14
87.57 16,718,580
26,285,486 63.78
100.27 29.347
62.798 29.33
35.62
YOLOv3 92.15 91.27 91.71 61,949,149 236.32 66.096 23.58
YOLOv4 93.32 91.56 92.44 64,363,101 245.53 60.334 19.97
YOLOv5s 93.30 As can be seen in Table
90.63 91.961: 7,276,605 27.76 17.060 58.18
YOLOXs 96.35 (1) 91.88
The mAP values94.11 8,968,255 showed 34.21
of DSMH-YOLOv4 different 26.806of
degrees 46.18
improvement
DSMH-YOLOv4 95.75 96.54 96.14 16,718,580 63.78 29.347 29.33
compared to the other six models, with a significant improvement of 11.08% and 8.57%
compared to Faster R-CNN and SSD, respectively, and 4.43% and 3.7% compared to the
YOLOv3 As can
andbeoriginal
seen in Table 1:
YOLOv4, and 4.18% and 2.03% compared to YOLOv5s and
(1) The mAP values of
YOLOXs, respectively. The algorithmDSMH-YOLOv4 showed
in this paper different
improved thedegrees of improvement
detection accuracy of
compared to the other six models, with a significant improvement
insulator breakage even more, reaching 96.54%, compared to YOLOv4 and YOLOXs; of 11.08% and 8.57%
the
improvement was 4.98% and 4.66%, respectively, indicating that the improved algorithmto
compared to Faster R-CNN and SSD, respectively, and 4.43% and 3.7% compared
inthe
thisYOLOv3
paper hasand original
a more YOLOv4,
accurate and 4.18%
localization effectand 2.03% compared
on insulators todefects.
and their YOLOv5s and
YOLOXs, respectively. The algorithm in this paper improved the
(2) From the comparison of the number of parameters and computation, we detection accuracy
can seeof
that the computation and number of parameters of the original YOLOv4 model are 60.334
G and 64,363,101 respectively, and the model size is 245.53 MB. There is a degree of diffi-
culty in deploying at the edge section. DSMH-YOLOv4 was lightened and improved to
reduce the number of calculations and parameters by 51.36% and 74.02%, respectively,
Electronics 2023, 12, 933 11 of 16

insulator breakage even more, reaching 96.54%, compared to YOLOv4 and YOLOXs; the
improvement was 4.98% and 4.66%, respectively, indicating that the improved algorithm in
this paper has a more accurate localization effect on insulators and their defects.
(2) From the comparison of the number of parameters and computation, we can see
that the computation and number of parameters of the original YOLOv4 model are 60.334 G
and 64,363,101 respectively, and the model size is 245.53 MB. There is a degree of difficulty
in deploying at the edge section. DSMH-YOLOv4 was lightened and improved to reduce
the number of calculations and parameters by 51.36% and 74.02%, respectively, and the
model size was reduced from 245.53 MB to 63.78 MB, indicating that the algorithm in this
paper can achieve the requirements of lightening and real-time detection.
(3) The detection speed of DSMH-YOLOv4 was 29.33 FPS, which is 46.87% higher
than the 19.97 FPS of the original YOLOv4 model. Although the detection speed is not as
fast as that of SSD, YOLOv5s, and YOLOXs, the detection accuracy has a clear advantage.
This indicates that DSMH-YOLOv4 can meet the requirements of deploying embedded
devices with a combination of both detection accuracy and speed.
Table 2 shows the results of the experiments for the current mainstream lightweight
YOLOv4 network.

Table 2. Comparison of evaluation indicators of mainstream lightweight YOLOv4 network.

Insulators AP Defects AP Model Size Speed

Model mAP (%) Parameters FLOPs (G)
(%) (%) (MB) (FPS)
MobilenetV1-YOLOv4 91.00 83.21 87.11 12,692,029 53.51 10,540 53.12
MobilenetV2-YOLOv4 90.66 87.68 89.71 10,801,149 48.70 8153 51.74
MobilenetV3-YOLOv4 91.36 87.12 89.25 11,729,069 56.3 7599 46.55
D-CSPDarknet53-YOLOv4 92.92 90.70 91.81 16,483,909 62.71 26.315 32.34
Xu et al. [6] 91.21 93.00 92.11 14,963,517 57.08 10,905 44.25
DSMH-YOLOv4 95.75 96.54 96.14 16,718,580 63.78 29,347 29.33

As can be seen in Table 2, based on the lightweight model, the YOLOv4 model using
D-CSPDarknet53 as the backbone network has the highest mAP value compared with the
YOLOv4 model under the Mobilenet series backbone network. The mAP of D-CSPDarknet53-
YOLOv4 is 4.7%, 2.1%, and 2.56% higher than that of MobilenetV1-YOLOv4, MobilenetV2-
YOLOv4, and MobilenetV3-YOLOv4, respectively. Although the detection speed of D-
CSPDraknet53-YOLOv4 is inferior to that of MobilenetV1-YOLOv4, MobilenetV2-YOLOv4,
and MobilenetV3-YOLOv4, the detection accuracy of D-CSPDraknet53-YOLOv4 has obvious
advantages. The improved algorithm proposed in [6] effectively improves the detection
accuracy of insulator defects, but the average detection accuracy is still low, and the mAP
value of the DSMH-YOLOv4 model is 4.03% higher than that of the improved algorithm
proposed in literature 6.
Table 3 shows the results of the ablation experiments for the three modified methods.

Table 3. Comparison of evaluation indicators of various improved algorithms.

Insulators Defects mAP Model Size FLOPs Speed

Model D-CSPDarknet53 SA 4Head Parameters
AP (%) AP (%) (%) (MB) (G) (FPS)
YOLOv4 √ 93.32 91.56 92.44 64,363,101 245.53 60.334 19.97
Algorithm 1 √ 92.92 90.70 91.81 16,438,909 62.71 26.315 32.34
Algorithm 2 √ 95.64 93.16 94.40 64,363,213 245.53 60.344 19.20
Algorithm 3 √ √ 94.65 94.14 94.39 65,423,452 249.57 70.762 15.60
Algorithm 4 √ √ 95.04 93.58 94.31 16,439,021 62.71 26.316 31.74
Algorithm 5 √ √ √ 94.53 94.82 94.67 16,718,460 63.78 29.346 30.97
DSMH-YOLOv4 95.75 96.54 96.14 16,718,580 63.78 29.347 29.33

√ add a shallow feature layer with 104 × 104 to the original three
SA: Shuffle Attention Neural Networks; 4Head:
feature detection layers. The position of “ ” in the table indicates that the improved algorithm adopts the
corresponding strategy.

As can be seen in Table 3:

(1) Algorithm 1 replaces the backbone network with D-DSPDarknet53 only; the model
size is compressed from 245.53 MB to 62.71 MB, the number of parameters is only 25.54%
Electronics 2023, 12, 933 12 of 16

of the original model, the computation is only 43.62% of the original model, the detection
speed is improved by 12.37 FPS, and the compression effect is significant, but the insulator
detection accuracy is decreased compared to that of the original YOLOv4 model by 0.4 %
and the recognition accuracy of the defect decreased by 0.86%.
(2) Algorithm 2 embeds the SA-Net module in the original YOLOv4 model, with almost
Electronics 2023,
Electronics
Electronics 12,
2023, x FOR
2023,
12, x12,
FOR noPEER
xPEER
FOR increase
REVIEW
PEER REVIEW
REVIEW in the number of parameters and computation, and no significant decrease 13 13ofin 16 16
13
of of 16
detection speed. The insulator detection accuracy increases by 3.06% compared to the
original YOLOv4 model, and the recognition accuracy of the defect increases by 2.8%.
(3) Algorithm
(3)(3) Algorithm 3 adds
Algorithm
(3) Algorithm 3aadds
3 adds 3shallow
a shallow
adds a feature
a shallowshallowfeature layer
feature to layer
layer
feature layer the
to to detection
the the
to detection
detection
the layer
detectionlayer oflayer
layer the
of of original
theof
theoriginal
original
the original
YOLOv4 YOLOv4model,
YOLOv4
YOLOv4 increasing
model, model,
model, increasing the
increasing
increasing number
thethe number
the of
number parameters
number of of parameters by
of parameters
parameters 1.65%
byby 1.65%
by of
1.65% the
of of
1.65% original
thethe
oforiginal
the model
original
original model
model andand
model andand
decreasing the detection
decreasing
decreasing
decreasing thethe detection
the speed
detection
detection by
speed
speed 4.37
byby
speed FPS,
4.37
by
4.37 but
FPS,
4.37
FPS, the
but
FPS,but insulator
the
but
the insulator
the
insulator detection
detection
insulator accuracy
detection
detection accuracy increases
accuracy
accuracy increasesincreases
increases
by 1.33%
byby compared
1.33%
by
1.33% compared
1.33% to the
compared
compared tooriginal
thethe
to tooriginal
the YOLOv4
original
original YOLOv4
YOLOv4 model,
YOLOv4 model, and
model,
model, the
and and defect
the
andthedefect
the recognition
defect recognition
defect accuracy
recognition
recognition accuracy accuracy
accuracy
increases by
increases
increases2.58%.
increasesbyby 2.58%.
by
2.58%.2.58%.
(4) The (4)results
(4)The
(4)
The ofresults
ablation
results
The of of
results experiments
ablation
of ablation
ablation experiments with Algorithms
experiments
experiments with
with Algorithms
with 3–5 show
Algorithms
Algorithms that
3, 3,4, 4,
and
3, D-DSPDarknet53
4, 5and
and show 5 show
5 show thatthat D-
thatD- D-
compresses a large number
DSPDarknet53
DSPDarknet53
DSPDarknet53 compresses
compresses of parameters
compresses a large
a large ofnumber
number
a large number theoforiginal
parameters
of parametersmodel,
of parameters and
of of
the SA-Net
oforiginal
the the
original improves
model,
original model, and
model, andthe
SA-
andSA-SA-
detection
NetNetaccuracy
improves
Net improves
improves of
thethe model
detection
the detection
detection and,
accuracywhile
accuracy
accuracy ofadding
thethe
of ofmodel
the more
model and,
model detection
and, while
and, while layers,
adding
while adding further
more
adding more improves
detection
more detection
detection lay-lay-lay-
ers,
ers,further
the model’s ers, detection
further improves
further improves
of small
improves thethemodel’s
the
targets. model’s
model’s detection
The detection
improved
detection of ofsmall
of targets.
small
DSMH-YOLOv4
small targets. The
targets. improved
TheThe shows improved
improved a DSMH-
2.43%
DSMH- DSMH-
YOLOv4 YOLOv4
YOLOv4
improvement shows
in theshows
shows a 2.43% a 2.43%
a 2.43%
detection improvement
improvement
improvement
accuracy of theinin the indetection
the the
insulator detection
detection accuracy
itself accuracy
and a of
accuracy thethe
of
4.98% ofinsulator
the
insulator
improvement itself
insulator itself
itselfand
inandand
a 4.98%
a 4.98%
the identification improvement
a 4.98% improvement
improvement
of small targets in in thethe
inidentification
the
for identification
identification
insulator of ofsmall
defects. small targets
of small
The targets
model forfor
targets insulator
for
insulator
compression defects.
insulator defects. The
defects. The
is 25.98% model
The ofmodel
model
compression
model,isand
compression
compression
the original 25.98%
is isthe25.98%
25.98% of of the
detection of
the original
theoriginal
speed model,
original
is model,
improved and
model, and the
and
bythe detection
the detection
detection
46.87% speed
speed
compared is to
speed improved
is is improved
improved
19.97 FPS byby by
46.87%
of YOLOv4,46.87% compared
46.87%
which compared
compared to to
is feasible 19.97
tofor
19.97 FPS
19.97 FPS of
FPS YOLOv4,
of
high-accuracy of YOLOv4,
YOLOv4, which
which
detection is is
which feasible
of isedge-end
feasible forfor
feasible high-accuracy
for high-accuracy
high-accuracy
insulator detection
faults. detection
detection
of of
Tableedge-end
of edge-end
edge-end
4 further insulator insulator
insulator
demonstrates faults. faults.
faults. the prediction results of each improved algorithm by
selecting three typical aerial images of the
Table
Table 4
Table further
4 4
further demonstrates
further demonstrates
demonstrates theprediction
the prediction
prediction
insulators. results
In Table results 3,ofthe
results each
of of
each improved
redeach improved
boxes arealgorithm
improved algorithm
algorithm
insulators byby by
selecting
selecting
selecting three
three typical
three typical aerial
typical aerial images
aerialimages of
images insulators.
of of
insulators.
and the blue boxes are defects. It can be seen from the recognition of Image 1 that when In
insulators. Table
In In
Table 3,
Table the
3, red
3,
the the
redboxes
redboxes are
boxes areinsulators
are insulators
insulators
there and
areandthe
andtheblue
obscured the
blue boxes
blue boxes
insulators areare
boxes defects.
are theItimage,
in defects.
defects. can
It Itbe
can can
beseen
only be
seen from
seenfrom
Algorithm thethe
from recognition
therecognition
2 with theoffusion
recognition Image
of of Image
Image 1 that
model 1 when
1 that that
when
can when
there
thereare
identify them, indicating that SA-Net is helpful in the recognition process for the obscured can
there areobscured
are obscured
obscured insulators
insulators
insulators in the
in in
theimage,
the
image, only
image, only Algorithm
only Algorithm
Algorithm 2 with
2 2
with the
withthefusion
the
fusion model
fusion model can
model can
identify
insulator identify
targets.them,
identify them, indicating
them,
From indicating
indicating thatthat
the recognition SA-Net
that
SA-Net is
SA-Netofishelpful
helpful
image in
is helpful the
2,in ittherecognition
incan
the
recognition
be seenprocess
recognition process
process
that whenforforthe
for
theobscured
therethe obscured
obscured
are
insulator
insulator targets.
targets. From From the the recognition ofimage
multiple insulators in the image and the size difference is too large, only Algorithmare
insulator targets. From recognition
the recognition of of
image 2,
image it
2, can
2,
it canitbecan seen
be be
seen that
seenthatwhen
that
when there
when there are
therearemul- 1 mul-
mul-
tiple insulators
tiple
tiple insulators
insulators in inthethe
inimage
the
image and
image and the
and size
the the
size difference
size difference
difference is istoo islarge,
too too
large, only
large,
only Algorithm
only Algorithm
Algorithm 1 with
1 with thethethe
1 with
with the fusion improvement can identify the insulators of small targets, indicating that
fusion
fusion improvement
fusion improvement
improvement can canidentify
can identify
identifythetheinsulators
the insulators
insulators of ofsmallofsmall targets,
small targets, indicating
targets, indicating
indicating thatthat D-D- D-
that
D-CSPDarknet53 does help in the recognition of small targets. From image 3 it can be
CSPDarknet53
CSPDarknet53
CSPDarknet53 doesdoes help
does help in in
help thethe
inrecognition
recognition
the recognition of ofsmall
small
of smalltargets.
targets. From
targets. From image
From image 3 it3 can
image it3can
itbe be
can seenseen
be seen
seen that the addition of a multiple output detection layer can be of great help in the
that
that
the
thattheaddition
addition
the additionof of a multiple
aofmultiple
a multiple output
output detection
output detection
detectionlayer
layer can
layer can be be
canofbe of
great
great help
of great help in in
helpthethe
inidentifi-
identifi-
the identifi-
identification of broken targets.
cation of of
cation broken
cation targets.
of broken
broken targets.
targets.
Table 4. TestTable
Table
Table
image
4. Test
4. Test
comparison.
4.image
Test comparison.
image
image comparison.
comparison.

Model
Model
Model
Model Image 1 1 11
Image
Image
Image Image 2 22 2
Image
Image
Image Image 33 3 3
Image
Image
Image

YOLOv4
YOLOv4
YOLOv4
YOLOv4

Algorithm 11 1 1
Algorithm
Algorithm
Algorithm

Algorithm
Algorithm
Algorithm2 2 2
 Electronics 2023, 12, 933 13 of 16

Algorithm
Algorithm 1 1 1
Algorithm

Table 4. Cont.

Model Image 1 Image 2 Image 3

Algorithm
Algorithm 22 2 2
Algorithm
Algorithm
Electronics 2023, 12, x FOR PEER REVIEW 14 of 16
Electronics
Electronics 2023,
Electronics
2023, 12,
2023,
12, x12,
FOR
x FOR PEER
xPEER
FOR REVIEW
PEER REVIEW
REVIEW 14 14
of of
16 16
14 of 16

Ours Ours
Ours
Ours Ours

ToTofurtherToTo
further further
To further
further
visualize
visualize visualize
visualize
visualize
thethe thethe
attention
attention attention
the
attention
region
region inregion
attention
region region
the
in in
the in the
the
model,
model, model,
inmodel,
the
thismodel,
this this
this
paper paper
this
paper
paper presents
paper
presents
presents
presents a fractional
apresents a fractional
a fractional
fractional
a fractional
heat map
heat map heat
heat mapmap
heat
visualizationvisualization
map
visualization visualization
visualization
analysis
analysis analysis
of ofanalysis
analysis
thethe of of the
theof
predicted predicted
the predicted
predicted
predicted results results
results
results
of of
results of
thethe of
thethe
of DSMH-YOLOv4
the DSMH-YOLOv4
DSMH-YOLOv4
DSMH-YOLOv4
DSMH-YOLOv4 model. model.
model.
model. As AsAsAs As
model.
cancan
bebe
seen can
can be
from
seen be
can seen
be
seen
from from
seen
from
Figure
Figure fromFigure
Figure
11,11, Figure
thethe11,11,
heat the
11,
the
heatmap heat
the
heat
map mapmap
heat
focuses mapfocuses
focuses
focuses
focuses onon on
thetheonthe
centerthe
on center
the
center
centerof thecenter
of
of the of the
theof
target target
the
target
target detection
target detection
detection
detection
detectionbox, box,
box,
box, box,
with with
with
thethe
with focal the
with
the
focalarea focal
the
focalfocalarea
area
areaatatthe at at
area
the
thecenter the
at center
the
center
centerofofthe center
of of
thetargetthethe
of
targetin target
the
target
inred in
red andin
target red
in
red andand
red and
a a decreasing
a decreasing
decreasing
and aa decreasing proportion proportion
proportion
proportion
proportion of ofof atten-
of
atten-
ofattention
atten- atten-
tiontion
tion
spreading spreading
spreading
spreading
outwards. outwards.
outwards.
outwards.
This further This
This Thisfurther
further
validates further validates
validates
validates
the model thethemodel
model
the model
improvements. improvements.
improvements.
improvements.
TheThegreen The
The green
The
green
boxes green
tion spreading outwards. This further validates the model improvements. green
boxes
boxes represent
boxes represent
represent detected
detected
detected insulators
insulators
insulators andandthethe
and orange
the
orangeorange boxes
boxes boxesrepresent
represent
represent detected
detected
detected defects.
defects.
defects.
represent
boxes representdetected insulators
detected insulatorsand andthe orange
the orange boxes represent
boxes represent detected
detecteddefects.
defects.

(a)(a) (a) (b)(b)(b)

(a) (b)

(c)(c) (c) (d)(d)(d)

(c) (d)

Figure 11. Cont.

Electronics 2023, 12, 933 14 of 16

(c) (d)

(e) (f)
Figure 11. Heat map of the detection of the DSMH-YOLOv4 model. (a) Seven insulators have
been detected; (b) four insulators and a defect have been detected; (c) six insulators have been
detected; (d) one insulator and two defects have been detected; (e) five insulators have been detected;
(f) five insulators and a defect have been detected.

5. Conclusions
This paper proposes an improved algorithm, DSMH-YOLOv4, for insulator and defect
detection based on YOLOv4. Firstly, the backbone feature extraction network is replaced
with D-CSPDarknet53 to significantly reduce the number of parameters and computational
effort in the model. In addition, an attention mechanism is embedded before the feature
fusion and the promiscuous feature layers are used to enhance the feature extraction
capability of the network. Finally, a large-size detection layer is added to enhance the
network’s ability to identify small targets for insulator defects.
The experimental results show that the number of parameters of the improved net-
work model is only 25.98% of that of the original model; the detection speed is improved by
9.36 FPS; the recognition accuracy of insulator defect is 96.54%, which is 4.98% higher than
that of the original YOLOv4 network; and the mean average accuracy of mAP for insulators
and their defects is improved from 92.44% to 96.14%. The improved algorithm signifi-
cantly improves the performance and provides an effective way to realize the embedded
application of real-time transmission line insulator location and defect recognition.

Author Contributions: Conceptualization, G.H. and Q.Y.; methodology, G.H.; software, Q.Y.; vali-
dation, Q.Y., R.W. and L.Z.; formal analysis, G.H.; investigation, F.Z. and S.L.; resources, S.Y.; data
curation, M.H., F.Z. and S.L.; writing—original draft preparation, Q.Y.; writing—review and editing,
G.H., Q.Y. and L.Q.; visualization, R.W.; supervision, S.Y. All authors have read and agreed to the
published version of the manuscript.
Funding: This work was supported by Young Talents Project of Scientific Research Foundation of
Education Department of Hubei Province with Grant No. Q20151601 and the National Key R & D
Program of China (No. 2020YFB0905900).
Data Availability Statement: Dataset link: https://github.com/InsulatorData/InsulatorDataSet
(accessed on 11 April 2022).
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Ru, C.; Zhang, S.; Zhang, Z.; Zhu, Y.; Liang, Y. Fault Identification Method for High Voltage Power Grid Insulator Based on
lightweight Mobilenet-SSD and MobileNetV2-DeeplabV3+ Network. High Volt. Eng. 2022, 48, 3670–3679.
2. Hao, S.; Ma, R.; Zhao, X.; Ma, X.; Wen, H.; An, B. Self-Explosion Fault Detection Algorithm for Glass Insulator Based on
Super-Resolution Deep Residual Network. High Volt. Eng. 2022, 48, 1817–1825.
3. Hao, Y.; Liang, W.; Yang, L.; He, J.; Wu, J. Methods of image recognition of overhead power line insulators and ice types based on
deep weakly-supervised and transfer learning. IET Gener. Transm. Distrib. 2022, 16, 2140–2153. [CrossRef]
4. Han, G.; He, M.; Gao, M.; Yu, J.; Liu, K.; Qin, L. Insulator Breakage Detection Based on Improved YOLOv5. Sustainability 2022,
14, 6066. [CrossRef]
Electronics 2023, 12, 933 15 of 16

5. Sadykova, D.; Pernebayeva, D.; Bagheri, M.; James, A. IN-YOLO: Real-Time Detection of Outdoor High Voltage Insulators Using
UAV Imaging. IEEE Trans. Power Deliv. 2020, 35, 1599–1601. [CrossRef]
6. Xu, S.; Deng, J.; Huang, Y.; Ling, L.; Han, T. Research on Insulator Defect Detection Based on an Improved MobilenetV1-YOLOv4.
Entropy 2022, 24, 1588. [CrossRef]
7. Han, G.; Li, T.; Li, Q.; Zhao, F.; Zhang, M.; Wang, R.; Yuan, Q.; Liu, K.; Qin, L. Improved Algorithm for Insulator and Its Defect
Detection Based on YOLOX. Sensors 2022, 22, 6186. [CrossRef]
8. Hao, K.; Chen, G.; Zhao, L.; Li, Z.; Liu, Y.; Wang, C. An Insulator Defect Detection Model in Aerial Images Based on Multiscale
Feature Pyramid Network. IEEE Trans. Instrum. Meas. 2022, 71, 3522412. [CrossRef]
9. Waleed, D.; Mukhopadhyay, S.; Tariq, U.; El-Hag, A.H. Drone-Based Ceramic Insulators Condition Monitoring. IEEE Trans.
Instrum. Meas. 2021, 70, 6007312. [CrossRef]
10. Li, B.; Zeng, J.; Zhu, X.; Wang, S.; Guo, Z.; Liu, H. Insulator defect detection network based on multi-scale context awareness.
High Volt. Eng. 2022, 48, 2905–2914.
11. Zhai, Y.; Yang, K.; Wang, Q.; Wang, Y. Disc Insulator Defect Detection Based on Mixed Sample Transfer Learning. Proceedings
of the CSEE. Available online: https://kns.cnki.net/kcms2/article/abstract?v=iJQOcxL8JgxGpwh6a5pyNkiV4NA4bKr6
Q3GvlsIcuvTz5BpQfttspv3LImqMB3lnLXSpS8_PW3DPTxm-wewaGlJFHjkUNfLnIhZWWaVPnwHbAiHgRMCK6Q==&
uniplatform=NZKPT&language=CHS. (accessed on 1 May 2022).
12. Saini, V.K.; Kumar, R.; Mathur, A.; Saxena, A. Short Term Forecasting Based on Hourly Wind Speed Data Using Deep Learning
Algorithms. In Proceedings of the 3rd International Conference on Emerging Technologies in Computer Engineering: Machine
Learning and Internet of Things ICETCE 2020, Jaipur, India, 7–8 February 2020; pp. 30–35.
13. Yang, Z.; Xu, Z.; Wang, Y. Bidirection-Fusion-YOLOv3: An Improved Method for Insulator Defect Detection Using UAV Image.
IEEE Trans. Instrum. Meas. 2022, 71, 3521408. [CrossRef]
14. Qiu, Z.; Zhu, X.; Liao, C.; Shi, D.; Qu, W. Detection of Transmission Line Insulator Defects Based on an Improved Lightweight
YOLOv4 Model. Appl. Sci. 2022, 12, 1207. [CrossRef]
15. Zhang, Z.; Zhang, B.; Lan, C.; Liu, H.; Li, D.; Pei, L.; Yu, W. FINet: An Insulator Dataset and Detection Benchmark Based on
Synthetic Fog and Improved YOLOv5. IEEE Trans. Instrum. Meas. 2022, 71, 6006508. [CrossRef]
16. Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Cheng-Yang, F.; Alexander, C. SSD: Single Shot Multibox Detector. In Proceedings
of the European Conference on Computer Vision—ECCV 2016, Amsterdam, The Netherlands, 11–14 October 2016; Springer:
Cham, Switzerland, 2016; pp. 21–37.
17. Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the
IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788.
18. Redmon, J.; Farhadi, A. YOLO9000: Better, Faster, Stronger. In Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 7263–7271.
19. Redmon, J.; Farhadi, A. Yolov3: An incremental improvement. arXiv 2018, arXiv:1804.02767.
20. Bochkovskiy, A.; Wang, C.Y.; Liao, H.Y.M. Yolov4: Optimal speed and accuracy of object detection. arXiv 2020, arXiv:2004.10934.
21. Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation.
In Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition IEEE Computer Society, Columbus, OH, USA,
23–28 June 2014; pp. 580–587.
22. Girshick, R. Fast R-CNN. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 7–13
December 2015; pp. 1440–1448.
23. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE
Trans. Pattern Anal. Mach. Intell. 2016, 39, 1137–1149. [CrossRef]
24. He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer
Vision, Venice, Italy, 22–29 October 2017; pp. 2961–2969.
25. Jia, X.; Yu, Y.; Guo, Y.; Huang, Y.; Zhao, B. Lightweight detection method of self-explosion defect of aerial photo insulator. High
Volt. Eng. 2022. [CrossRef]
26. Liu, J.; Liu, C.; Wu, Y.; Xu, H.; Sun, Z. An improved method based on deep learning for insulator fault detection in diverse aerial
images. Energies 2021, 14, 4365. [CrossRef]
27. Zhu, Y.; Zheng, Y.; Qin, J. Insulator target detection based on improved YOLOv3. Insul. Surge Arresters 2022, 3, 166–171.
28. Yang, L.; Fan, J.; Song, S.; Liu, Y. A light defect detection algorithm of power insulators from aerial images for power inspection.
Neural Comput. Applic. 2022, 34, 17951–17961. [CrossRef]
29. Yang, Z.; Xu, X.; Wang, K.; Li, X.; Ma, C. Multitarget detection of transmission lines based on DANet and YOLOv4. Sci. Program.
2021, 2021, 6235452. [CrossRef]
30. He, H.; Huang, X.; Song, Y.; Zhang, Z.; Wang, M.; Chen, B.; Yan, G. An insulator self-blast detection method based on YOLOv4
with aerial images. Energy Rep. 2022, 8, 448–452. [CrossRef]
31. Gao, W.; Zhou, C.; Guo, M. Insulator defect identification via improved YOLOv4 and SR-GAN algorithm. Electr. Mach. Control
2021, 25, 93–104.
32. Han, G.; He, M.; Zhao, F.; Xu, Z.; Zhang, M.; Qin, L. Insulator detection and damage identification based on improved lightweight
YOLOv4 network. Energy Rep. 2021, 7, 187–197. [CrossRef]
33. Zhang, Q.; Yang, Y. SA-Net: Shuffle attention for deep convolutional neural networks. arXiv 2021, arXiv:2102.00240.
Electronics 2023, 12, 933 16 of 16

34. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. arXiv 2015, arXiv:1512.03358.
35. Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely Connected Convolutional Networks. In Proceedings of the
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 2261–2269.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.