PCB DEFECT DETECTION

You-Only-Look-Once Multiple-Strategy
Printed Circuit Board Defect Detection Model
Xin Wang , Hongyan Zhang , Qianhe Liu , Wei Gong , and Song Bai , Shenyang Jianzhu University,
Shenyang, 110168, China
Hezhen You , Tongji University, Shanghai, 200092, China

Addressing the challenges of complex backgrounds, minute defects, and irregular

shapes in PCB defect images that often lead to missed detections, inaccurate
localizations, and false positives, this article introduces an improved you only look once
(YOLO) model, termed YOLO-Biformer, to enhance the network’s ability to detect surface
defects on PCBs. First, YOLO-Biformer incorporates a hybrid attention module to
differentiate the importance among various channels, thus strengthening the extraction
of small target defect features and preventing the loss of minor target information often
caused by deep convolutional networks. Second, the model introduces a jumping hollow
space convolutional pyramid aimed at preserving more image details and interrelated
information, thereby boosting the network’s defect localization capability. Finally, the
Enhanced Intersection Over Union-FocalLoss loss function is employed to enhance the
network’s ability to distinguish between defects with similar features. Experimental
results demonstrate that the proposed algorithm increases the mean average precision
by 4.1%, showing excellent performance in recognizing small target defects on printed
circuit board surfaces and achieving both high accuracy and real-time capabilities.

W
ith the development of the electronics open_circuit, mouse_bite, and missing_hole can occur
manufacturing industry, electronic prod- on the PCB surface. These defects significantly impact
ucts are evolving toward multifunctionality, the performance of PCB boards, leading to anomalies
intelligence, and miniaturization. As a crucial precision and safety incidents in the entire circuit system. There-
component of electronic products, the printed circuit fore, efficient PCB surface defect detection technology
board (PCB) plays a vital role, carrying the layout of cir- has become a critical issue that needs to be addressed
cuit components and wires. The quality of the PCB and in the PCB industry.20
its soldered components is critical to the performance Currently, several detection methods that meet
of the product. The rapid development of integrated cir- industrial production requirements have been researched
cuit packaging technology has led to electronic prod- and developed for different types of PCB defects. These
ucts becoming increasingly compact and lightweight. methods include manual visual inspection, inline instru-
This, in turn, has resulted in more crowded wiring on ment detection, functional testing, and visual inspec-
PCBs, thereby raising the quality standards for PCBs. tion methods. The last of these includes automated
To ensure the performance of electronic devices, optical inspection, machine vision inspection, and auto-
PCB defect detection technology has become a key mated visual inspection. Although these methods have
technique in the modern electronics industry. How- resolved defect detection to some extent, they are no
ever, due to its complex production process and longer sufficient to meet the needs of modern indus-
susceptibility to environmental and manual factors, tries. They have various subjective limitations, such as
various defects such as spurious_copper, spur, short, fatigue, speed, and cost; inline detection methods are
cost-effective and easy to operate but time-consuming
in programming and debugging; and functional testing
1070-986X © 2024 IEEE
Digital Object Identifier 10.1109/MMUL.2024.3359267
methods require specialized testing equipment in the pro-
Date of publication 30 January 2024; date of current duction line and at the end, with complex programming
version 8 April 2024. that is difficult to generalize.18 In view of this, this

76 Authorized licensed
IEEE use limited to: Zhejiang University. Downloaded
MultiMedia on July
Published by 28,2024
the IEEE at 15:05:37
Computer UTC from IEEE Xplore. Restrictions
Society January-Marchapply.2024
 PCB DEFECT DETECTION

article targets six common types of PCB defects, Object Detection Algorithms
addressing issues like complex surface textures and With the advent of deep learning, particularly the suc-
small defect targets. It selects the you only look once, cessful application of convolutional neural networks
version 8 (YOLOV8) algorithm, which balances detec- (CNNs), the field of object detection has undergone a
tion speed and accuracy, as the backbone network transformation. In 2015, Redmon et al.1 introduced
and proposes a PCB defect detection algorithm based the one-stage algorithm YOLO, transforming object
on a fusion of a hybrid attention mechanism and jump- detection into a single regression problem, mapping
ing hollow space convolutional pyramid (JHSCP). This directly from image pixels to bounding box coordinates
algorithm achieves high precision and efficiency in and class probabilities. Unlike traditional two-stage
detecting surface defects on PCBs, effectively meeting methods, like the regions with convolutional neural
the practical industrial production needs. The main networks (R-CNN) series,2,6 one-stage algorithms pre-
contributions of this article are as follows: dict object categories and locations directly from the
input image without generating candidate regions,
Introduction of a hybrid attention module (HAM)
thereby improving detection speed. However, these
to differentiate the importance between different
networks have a higher miss rate for small objects. To
channels, highlighting key information in shallow
enhance detection accuracy for small objects, Cha
networks and suppressing irrelevant information,
et al.13 improved traditional CNNs, achieving an accu-
thus enhancing the algorithm’s performance in
racy of 98% for small object detection but requiring
detecting small targets.
4.55 s to detect an image.
Replacement of the original scale prediction
Single shot multibox detector (SSD) balances preci-
feature fusion (SPFF) module with a jumping
sion and speed by introducing multiscale feature
hollow space convolutional pyramid module
maps, improving small object detection.3 RetinaNet
(JHSCP), which increases the convolution ker-
introduced focal loss to address the common issue
nel’s field of reception while preserving more
of foreground–background class imbalance in object
image details and enhancing information rele-
detection and utilized Feature Pyramid Network to effi-
vance, significantly improving the accuracy of
ciently detect objects of various scales.8 FasterNet-SSD,
locating minute defects.
using a FasterNet network built on partial convolution
In view of the difficulty in distinguishing PCB
instead of the original SSD model’s VGG16 backbone,
defect types, the Enhanced Intersection Over
Union (EIOU)-FocalLoss loss function is intro- achieved an mean average precision (mAP) of 80.38%
duced to calculate classification errors, adjust on small object test sets. However, these introductions
the network’s focus on defects with similar fea- have slowed down the network detection speed.7
tures, and enhance the recognition accuracy of YOLOV7, through improved network architecture,
the network model. has enhanced training efficiency, achieving an mAP
of 87.51% on small object test sets with 36% lower
The structure of this article is arranged as follows: computational demand compared to its predecessor.
First, the “Related Work” section reviews research However, it struggles to accurately detect multiscale
related to the improved YOLOV8 model proposed in objects.4 Recently, Reis et al.5 utilized anchor-free
this article. Then, the structure of the YOLOV8 model is detection and online data augmentation in YOLOV8 to
detailed. Afterward, the “Experiments” section introdu- improve inference speed and training efficiency. Never-
ces the dataset used and the experimental setup as theless, due to the working principle of convolution,
well as analyzes and discusses the results. Finally, the the network’s accuracy for small object detection still
“Conclusion and Future Work” section summarizes the falls short of industrial needs.5 Therefore, this article
article and looks forward to future research directions. introduces an attention mechanism to simultaneously
satisfy the precision and speed requirements for detect-
RELATED WORK ing minute PCB surface defects.
Defect detection is a critical issue in the field of
machine vision and has garnered significant attention Attention Mechanism
from researchers. In this section, we review works The attention mechanism focuses on key information
related to our proposed improvements to the YOLOV8 and improves feature representation by learning atten-
model, encompassing object detection algorithms, tion weights through specific networks and then apply-
attention mechanisms, and dilated convolutional ing these weights to the learned features. Currently,
pyramids. attention mechanisms can be categorized into three

Authorized licensed
January-March 2024 use limited to: Zhejiang University. Downloaded on July 28,2024 at 15:05:37 UTC from IEEE
IEEE Xplore. Restrictions apply.
MultiMedia 77
PCB DEFECT DETECTION

types: channel attention, spatial attention, and adap- has been integrated into the YOLOV8 framework, lead-
tive attention. ing to the development of the YOLO-Biformer network.
The channel attention mechanism, focusing on Initially, a HAM that processes a small subset of rele-
deep features of colors and features by weighting dif- vant markers in a query-adaptive manner is introduced.
ferent channel responses,9,10 slows down the overall This enables the network to focus without interference
network detection speed due to its complex structure. from irrelevant markers. Subsequently, the JHSCP
Spatial attention mechanisms, typically used to analyze replaces the SPFF, enhancing the network’s capability
local regions of an image and guide the model to to detect minute targets. Finally, the EIOU-FocalLoss
focus on certain areas,11 offer reduced computational loss function is employed to increase the network’s
requirements compared to channel attention but show convergence speed and detection precision. The net-
limited improvement in small object detection accuracy. work is configured to process input images of 640

Unlike the previous two, adaptive attention mechanisms 640 pixels, utilizing three HAMs inserted at different
dynamically shift focus based on the specific content of points in the network to accommodate the detection
input data.12 In PCB surface defect detection tasks, of targets of various sizes. The specific network struc-
although the defects are small, there are significant ture is illustrated in Figure 1.
structural differences between defects and back-
ground information. Enhancing the network extrac- HAM
tion of these differences with adaptive attention In YOLOV8, the backbone utilizes Conv2d and c2f mod-
mechanisms can ensure network speed while improv- ules for downsampling, which results in a certain loss
ing detection accuracy. of contextual features, leading to reduced resolution of
feature maps and loss of spatial information about
Dilated Convolutional Pyramid PCB surface defects. To mitigate the loss of informa-
The dilated convolutional pyramid enhances the net- tion about small targets during the downsampling pro-
work’s spatial awareness while maintaining the same cess, an attention mechanism is employed to enhance
number of parameters, focusing on capturing multi- the network’s focus on small targets.
scale information in images. In the SDDNet network, it Compared to the classic Squeeze-and-Excitation
has been applied for segmenting concrete cracks with attention mechanism and Convolutional Block Attention
an accuracy of 84.6%, but it currently does not meet Module, the adaptive attention mechanism Biformer, as
shown in Figure 2, is capable of dynamically adjusting
PCB detection requirements.14 PSNet improved accu-
the weights of channels and structures while reducing
racy by 10% compared to other models through innova-
the number of computational parameters. Biformer
tive encoder–decoder architecture and adjustment
maps the given 2-D input features X 2 RH
W
S , the
of the dilation rate, but its detection speed is not
mechanism partitions it into nonoverlapping regions of
sufficient for real-time detection.17 Kang and Cha15
size S
S, with each region encapsulating HW =S 2 fea-
introduced an attention mechanism into deep CNNs,
ture vectors. The next step involves reshaping X into
improving detection accuracy to 92.6% on split seg-
X 2 RS
HW =S
C , yielding tensors Q, K, and V through
2 2

mentation datasets and achieving a speed of 49.2

linear mapping:
frames per second (FPS). AGAN introduces an attention
mechanism into the generative adversarial network net- Q ¼ Xr W q (1)
work to generate data for segmentation networks, K ¼ Xr W k (2)
resulting in a segmentation accuracy of 93.15% for the V ¼ Xr W v : (3)
IDSNet network. However, stable training of AGAN is
The next step involves the establishment of a directed
challenging, and the addition of the attention mecha-
graph to ascertain the interregion attentional connec-
nism increases computational complexity.16
tivity. This facilitates the derivation of region-level
Qr , K r 2 RS
C through the computation of the mean
2

YOLO-BIFORMER PCB DEFECT of Q and K for each region. Subsequently, the trans-
DETECTION ALGORITHM posed forms of Qr and K r undergo matrix multiplica-
This section explains the distinctions between the tion to yield the connectivity matrix of the region and
improved YOLOV8 and the original network and dis- region affinities, referred to as Ar 2 RS
S .
2 2

cusses the impact of the improved modules on defect

detection accuracy and speed. Addressing the issues Ar ¼ Qr ðK r ÞT : (4)
mentioned earlier and considering the characteristics The entries within the adjacency matrix Ar serve as
of the PCB defect dataset, an attention mechanism quantifications of the semantic interconnectedness

78 Authorized licensed
IEEEuse limited to: Zhejiang University. Downloaded on July 28,2024 at 15:05:37 UTC from IEEE Xplore. Restrictions
MultiMedia January-Marchapply.2024
 PCB DEFECT DETECTION

FIGURE 1. YOLO-Biformer network structure. Conv2d: convolutional 2D; k: kernal; s: stride; p: padding; C2F: convolutinal
to fusion module.

between two regions. To distill the most relevant article constructs a HAM. In the existing vertical net-
connections for each region, the pivotal procedure work, two additional branches are added: one adjusts
involves retaining only the top k most closely associ- the feature channels through 1
1 convolution, and
ated regions. This process can be encapsulated as fol- the other inputs features into the attention layer after
lows, wherein the routing index matrix I r 2 N S
K 3
3 convolution. The output from the attention layer
2

interfaces with the line-by-line topk operator: is directly merged with the channels, while the other
branch first undergoes convolution to reduce the num-
I r ¼ topIndexðAr Þ: (5)
ber of parameters and then merges the channels. After
However, a single-layer attention mechanism can each convolution, batch normalization is not added,
only improve specific parts of the network. To enhance and a linear activation function is used to avoid dis-
the overall performance of the entire network, this rupting the interest propagation.19 Finally, the attention

FIGURE 2. Biformer structure.

Authorized licensed
January-March 2024 use limited to: Zhejiang University. Downloaded on July 28,2024 at 15:05:37 UTC from IEEE
IEEE Xplore. Restrictions apply.
MultiMedia 79
PCB DEFECT DETECTION

FIGURE 3. Structure diagram of the HAM. concat: concatenate.

mechanism channels and downsampling structures

are combined to produce the output of the attention
mechanism module. The Biformer module is illustrated
in Figure 3.

JHSCP
PCB surface defects are often small targets, among
which mouse bite is extremely small and similar in color
to the background. The SPFF structure of YOLOV8,
which uses a multilayer maximum pooling structure, can
affect the accuracy of the detection network when the
target is small, and features, such as shape and color,
are not distinct. To address this issue and enhance the
network’s ability to detect minute targets, thereby bet-
ter extracting PCB surface defect features, a solution
is needed. In previous work, the dilated spatial convo-
lutional pyramid has been utilized to address large
object-scale variations in the segmentation field. How-
ever, a large dilation coefficient in spatial convolution
is not conducive to detecting minute defects on the
PCB surface.
To extract minute features on the PCB surface, this
article adopts a jumping hollow space pyramid pooling
module composed of spatial convolutions with dilation
rates of one, two, five, one, two, and five and a convolu-
tional kernel size of 3
3. This configuration helps to
avoid the local information loss caused by maximum
pooling. To reduce computational complexity, a maxi-
mum pooling layer with fewer feature parameters is
introduced after the dilated convolution. In the JHSCP
module, the input feature map is processed through
two convolutions and then sampled into two indepen-
dent channels. One channel undergoes no additional
processing, while the other is layered through dilated
convolutions with different expansion coefficients to FIGURE 4. Structure diagram of JHSCP.
extract feature maps containing different receptive
fields. The structure of this module is illustrated in
Figure 4. The effect of the null rate on the sensory
Here, k denotes the convolutional kernel size, while r
field of the convolutional layer is given in the follow-
signifies the null rate. From the equation, it is evident
ing equation:
that the receptive field’s size is directly proportional
R ¼ ðr  1Þ
ðk  1Þ þ 1: (6) to the null rate. Consequently, a larger receptive

80 Authorized licensed
IEEEuse limited to: Zhejiang University. Downloaded on July 28,2024 at 15:05:37 UTC from IEEE Xplore. Restrictions
MultiMedia January-Marchapply.2024
 PCB DEFECT DETECTION

field equates to heightened information capture. With center point of the frame, Ldis for the frame’s edge
this in mind, the JHSCP module serves to address the length, b, bgt for the coordinates of the center point
information loss incurred by the SPP structure. This shared by the prediction frame and target frame, and
enhancement bolsters information coherence, simulta- xgt , hgt denoting the width and height of the minimum
neously safeguarding a greater array of details about enclosing boundary for both frames. The adjustment of
small targets. The overarching aim is to attain a more x tailors the convergence speed and ultimate accuracy
precise small target detection capability. for smaller IOU samples. In this article, x assumes a
value of 0.5, thereby markedly enhancing the conver-
EIOU and Focal Loss Function gence speed and detection precision of network learn-
PCB surface defects occupy a small proportion of the ing in a well-justified manner.
entire image. The YOLOV8 network employs a combi-
nation of complete intersection over union (CIOU) and EXPERIMENTS
distribution focal loss (DFL) for loss calculation. CIOU This section conducts experiments on the YOLO-
emphasizes the aspect ratio difference between two Biformer algorithm using publicly available datasets.
bounding boxes, but, given the small size and less pro- First, the datasets used for the experiments are intro-
nounced aspect ratio differences of PCB surface duced. Next, key metrics for evaluating the model are
defects, EIOU loss, unlike CIOU, also considers the size discussed. Subsequently, a comprehensive experimen-
and aspect ratio of bounding boxes, ensuring that the tal analysis is conducted on the factors influencing
predicted box shape closely matches the actual box, the detection performance of the YOLO-Biformer algo-
even for small targets. Meanwhile, DFL focuses on rithm, including the loss function and network struc-
improving the prediction of bounding box positions, ture. Finally, a comparison is made between the
especially in cases where the boundaries of detected proposed algorithm and the original YOLOV8 algo-
objects are blurred or hard to discern. rithm. All tests are conducted on a single computer,
In PCB defect detection tasks, the features of
with specific configurations as shown in Table 1.
defects like spur, mouse bite, and spurious copper are
quite similar, requiring a loss function that pays more
Introduction to Datasets
attention to samples with hard-to-distinguish features.
This experiment utilizes the PCB defect dataset
The Focal-Tversky loss function has been applied in
released by Peking University, as illustrated in Figure 5.
image segmentation, but it necessitates the introduc-
The data, derived from the PCB dataset, consist of
tion of more hyperparameters, increasing computa-
images from an industrial production environment.
tional load and requiring task-specific hyperparameter
They includes 1386 image samples covering six differ-
adjustments.
ent categories of PCB defects: spurious copper, spur,
This article adopts a combination of EIOU-
short, open circuit, mouse bite, and missing hole. The
FocalLoss, dynamically adjusting the weight coeffi-
number and size of these defects are detailed in Table 2.
cients between focal loss and EIOU loss, allowing the
network to focus on different aspects at various stages
of training. This approach ensures that the network Model Evaluation Metrics
enhances its ability to detect minute defects on the This articles selects the average IOU threshold of 0.5
PCB surface while reducing the computational cost for the predicted bounding boxes compared to the tar-
of the loss function. The formula for EIOU-Focal Loss is get boxes. IOU represents the overlap rate between
as follows:
LFocal - EIOU ¼ IOUc LEIOU (7) TABLE 1. Computer configuration table.*

LEIOU ¼ LIOU þ Ldis þ Lasp (8) Configuration Version

q2 ðb, bgt Þ q2 ðx, xgt Þ q2 ðh, hgt Þ System Windows 10
LEIOU ¼ 1  IOU þ þ þ :
c2 Cx 2 Ch 2 RAM 64 GB
(9)
CPU COREI9-12900H
Incorporating the intersection over union (IOU)
GPU NVIDIA RTX 3060
metric, where IOU signifies the intersection ratio, the
parameter x is introduced as a hyperparameter regu- Deep learning framework PyTorch and CUDA
lating the curvature of the curve. The loss components version 11.8
include LEIOU for the intersection ratio, Ldis for the *RAM: random-access memory.

Authorized licensed
January-March 2024 use limited to: Zhejiang University. Downloaded on July 28,2024 at 15:05:37 UTC from IEEE
IEEE Xplore. Restrictions apply.
MultiMedia 81
PCB DEFECT DETECTION

FIGURE 5. PCB defect dataset.

the detection result (DR) bounding boxes and the enhances the overall performance of the model to
ground-truth (GT) bounding boxes. The evaluation met- varying degrees:
rics for the model include average precision (AP), mAP,
and FPS. Precision denotes the proportion of correctly Incorporating the HAM into the backbone net-
predicted samples among all targets. AP is the area work, model 2 significantly enhances the net-
enclosed by the precision–recall (P–R) curve and the work’s ability to characterize key features. This
x-axis, and its calculation formula is as shown in the pivotal upgrade effectively resolves the chal-
following: lenges posed by intricate background textures
in PCB images and the loss of intricate details
TP
Precision ¼ (10) during network propagation. To further validate
TP þ FP
TP the model’s superiority, the P–R diagram is
Recall ¼ (11) introduced, depicted in Figure 6. A comparison
TP þ FN
ð1 of the P–R diagrams reveals that the attention
AP ¼ ðPðRÞdRÞ (12) mechanism’s integration amplifies the network’s
0
X accuracy in recognizing small targets. This enhance-
ðAPÞ ment elevates the algorithm’s mAP to 98.4%,
mAP ¼ (13)
n thereby substantially augmenting network detec-
GT \ DR tion precision.
IOU ¼ (14)
GT [ DR In model 3, the SPFF module within YOLOV8 is
where TP indicates true positive, FP is false positive, substituted with the JHSCP module, depicted
and FN is false negative. in Figure 7. This transition leads to improve-
ments across various APs, resulting in a notewor-
Ablation Experiment: Module Analysis thy 0.19 enhancement in mAP. By expanding the
To thoroughly analyze the superiority of each module network’s sensory field, this module notably
in the improved YOLOV8 for PCB defect detection, this enhances the capability to detect instances of
article designs ablation experiments based on the origi- “mouse_bite” and” spur” defects. Utilizing spatial
nal YOLOV8. The enhanced backbone network, JHSCP convolutions with varying null rates, this modifi-
module, and EIOU-FocalLoss function are individually cation minimizes feature loss as compared to the
integrated into YOLOV8 for comparative experiments original YOLOV8 algorithm. Consequently, more
against the original network. The specific experimental intricate features are retained, bolstering feature
test results are presented in Table 3. By analyzing the extraction efficacy for diminutive targets like
contribution of each improvement strategy to the net- “short” and “spurious_copper.” This systematic
work in Table 3, it is observed that each module improvement significantly boosts the network’s
overall performance.
TABLE 2. Printed circuit board surface defect images. In model 4, the integration of the EIOU-
FocalLoss loss function as the classification
Number of error leads to a substantial enhancement in
Defect Name Images Image Size
mAP value, ascending from 94.8% to 96.8%, as
spurious_copper 231 3034
1586 showcased in Figure 8. This augmentation is
spur 231 3034
1586 particularly pronounced for challenging instances,
such as edge-blurred “spur” and “mouse_bite”
open_circuit 231 3034
1586
defects. The inclusion of the focus parameter in
mouse_bite 231 3034
1586 the loss function contributes to the escalation of
missing_hole 231 3034
1586 loss values for such intricate samples. Moreover,
for minute “short-circuit” and “broken-circuit”
short 231 3034
1586
defects, the balance parameter escalates the

82 Authorized licensed
IEEEuse limited to: Zhejiang University. Downloaded on July 28,2024 at 15:05:37 UTC from IEEE Xplore. Restrictions
MultiMedia January-Marchapply.2024
 PCB DEFECT DETECTION

penalty for prediction inaccuracies. These com-

mAP of
0.5 (%)
bined strategies culminate in an overall improve-

98.9
98.4

96.8
94.8

96.7
ment in detection accuracy. By conducting
thorough ablation experiments and juxtaposing
the results, a noticeable pattern emerges. As
missing_hole

95.9 illustrated in Figure 9, the most noteworthy

88.0

97.3
99.7
92.1
enhancement arises from model 5, the refined
YOLOV8 framework introduced in this study. The
amalgamation of diverse enhancement modules
yields a substantial performance surge. This
mouse_bite

algorithm notably excels in detecting defects

98.6
99.5

99.5
98.3
96.2

with indistinct edges and diminutive targets fea-

turing minimal pixel occupancy. Impressively, all
six categories of defective targets are aptly
identified, achieving an average accuracy aug-
open_circuit

mentation of 4.1%. This resoundingly affirms the

*AP: average precision; JHSCP: jumping hollow space convolutional pyramid; mAP: mean average precision; YOLOV8: you only look once, version 8.
96.6
99.0
98.8
99.3

99.5
AP of 0.5 (%)

efficacy of the proposed algorithm in the realm

of PCB surface defect recognition.
short

96.6
99.0
93.8
98.3
90.5

Testing Images and Discussions

To further verify the superiority of the improved
YOLOV8 algorithm proposed in this article, it is com-
pared with classic algorithms in the field of defect
spur
92.6

98.9
98.0

94.8
95.7

detection. This includes comparisons with two-stage

detectors, which are effective in detecting small tar-
gets, such as a region based fully convolutional neural
spurious_copper

network (R-FCN) and Faster R-CNN, as well as one-

stage detectors like SSD, YOLOV5, YOLOV7, and
99.5
99.5
99.3
99.5
99.5

YOLOV8. The comparative experiments use an mAP of

0.5 and FPS as the evaluation metrics for each algo-
rithm. Model training and testing are conducted using
the same hyperparameters on the dataset of this
EIOU-Focal

article. The detection results are listed in Table 4, and

Loss

Figure 10 presents the defect recognition performance

X
X

of different algorithms.
TABLE 3. Ablation experiments with modified YOLOV8. *

Analysis of Table 4 and Figure 5 indicates that

R-FCN and Faster R-CNN, as representative two-stage
JHSCP

algorithms, demonstrate certain detection capabilities

X

for PCB surface defects. Except for a few missed detec-

tions of minor defects, the prediction boxes for each
type of defect are generally accurate in both location
and size, and the categorization is correct. However,
HAM

their real-time performance falls far short of practical

production needs. Single-stage algorithms like YOLOV5
and YOLOV7 show some speed improvement com-
YOLOV8

pared to Faster R-CNN but suffer a significant drop in

X
X
X
X
X

accuracy.
The YOLOV8 algorithm has substantially improved
overall performance compared to its previous versions,
Number

with a 15.4% increase in speed over YOLOV7 and a 1.4%

Model

higher accuracy than Faster R-CNN. YOLOV8-Efficient,

4
3

5
2
1

an improvement on the YOLOV8 algorithm, raised the

Authorized licensed
January-March 2024 use limited to: Zhejiang University. Downloaded on July 28,2024 at 15:05:37 UTC from IEEE
IEEE Xplore. Restrictions apply.
MultiMedia 83
PCB DEFECT DETECTION

FIGURE 6. Comparison of curves with the introduction of Biformer. mAP: mean average precision; P–R: precision–recall; YOLOV8:
you only look once, version 8.

FIGURE 7. Comparison of P–R curves with the introduction of JHSCP.

FIGURE 8. Comparison of P–R curves with the introduction of EIOU-FocalLoss.

84 Authorized licensed
IEEEuse limited to: Zhejiang University. Downloaded on July 28,2024 at 15:05:37 UTC from IEEE Xplore. Restrictions
MultiMedia January-Marchapply.2024
 PCB DEFECT DETECTION

FIGURE 9. Comparison of P–R curves of YOLOV8 and YOLO-Biformer. R-CNN: region-based convolutional neural network.

TABLE 4. Comparison of detection results of different achieved the intended focus on small targets. The
algorithms. introduction of the EIOU-FocalLoss function effectively
reduces false positives and missed detections and,
Model mAP of 0.5 (%) FPS
to some extent, increases the network’s operational
R-FCN 88.7 21 speed. Compared to similar algorithms, it improves
Faster R-CNN 93.4 44 accuracy without a significant decrease in speed,
proving the exceptional performance of YOLO-
SSD 80.1 40
Biformer.
YOLOV5 91.3 46
YOLOV7 91.8 52 CONCLUSION AND
YOLOV8 94.8 60
FUTURE WORK
To address the issues of missed detections, inaccurate
YOLOV8- 95.6 47
Efficient localization, and false positives in the YOLOV8 algo-
rithm for PCB surface defect detection, this article pro-
YOLO-Biformer 98.9 52
poses a PCB defect YOLO detection algorithm based
on the fusion of attention mechanisms and dilated spa-
model’s mAP from 94.8% to 95.6%. However, as tial convolution pyramids. It achieves precise localiza-
observed in Figure 5, it still fails to detect some minor tion and recognition of PCB surface defects. In this
defects, like mouse bites on the PCB surface, not meet- model, a HAM is introduced into the main network to
ing the industrial production requirements for PCB enhance the network’s representation capability. The
boards. The proposed YOLO-Biformer achieves the jumping hollow convolution is combined with improved
highest mAP among these four algorithms, showing spatial pooling pyramids, increasing the network’s
improvements of 10.2%, 5.1%, and 4.3% over R-FCN, receptive field while effectively retaining image detail
YOLOV8, and YOLOV8-Efficient, respectively. As seen features. The Focal-EIOU Loss function is introduced
in Figure 5, the prediction boxes of YOLO-Biformer to calculate classification errors and adjust the net-
most closely match the actual position and size distri- work’s attention to feature-similar defects, impacting
bution of the defects, with the highest confidence. It detection accuracy.
effectively detects defects that other algorithms strug- Experimental results show that the proposed model
gle with, achieving satisfactory results. While YOLO- achieves an mAP of 98.9% on a public dataset while
Biformer shows a slight decrease in FPS compared to maintaining an FPS of 52. It can accurately and quickly
the original YOLOV8, it still significantly outperforms detect PCB surface defects with low contrast, varying
Faster R-CNN, meeting the high-speed, high-accuracy morphology, and small size. Compared to several com-
requirements for PCB production. monly used models, the YOLO-Biformer algorithm in
The combined results indicate that the integration this article demonstrates significant advantages in
of YOLOV8 with the HAM and JHSCP has successfully overall performance, meeting the speed and accuracy

Authorized licensed
January-March 2024 use limited to: Zhejiang University. Downloaded on July 28,2024 at 15:05:37 UTC from IEEE
IEEE Xplore. Restrictions apply.
MultiMedia 85
PCB DEFECT DETECTION

FIGURE 10. Comparison experiment effect diagram.

requirements of PCB surface inspection in practical 2. R. Girshick, J. Donahue, T. Darrell, and J. Malik,
production. Data are one of the crucial factors for “Rich feature hierarchies for accurate object
improving the detection accuracy of neural network detection and semantic segmentation,” in Proc.
models. Current data augmentation methods do not IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2014,
yield satisfactory results for improving the detection of pp. 580–587.
small targets. In the next step, we will focus on data 3. W. Liu et al., “SSD: Single shot MultiBox detector,”
augmentation methods for small targets, allowing the in Proc. 14th Eur. Conf. Comput. Vis. (ECCV), 2016,
network to be widely applied in the real-time detection pp. 21–37, doi: 10.1007/978-3-319-46448-0_2.
of various types of small PCB defects. 4. C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao,
“YOLOv7: Trainable bag-of-freebies sets new
state-of-the-art for real-time object detectors,” in
REFERENCES Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit.,
1. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You 2023, pp. 7464–7475.
only look once: Unified, real-time object detection,” in 5. D. Reis, J. Kupec, J. Hong, and A. Daoudi, “Real-time
Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., flying object detection with YOLOv8,” 2023,
2016, pp. 779–788. arXiv:2305.09972.

86 Authorized licensed
IEEEuse limited to: Zhejiang University. Downloaded on July 28,2024 at 15:05:37 UTC from IEEE Xplore. Restrictions
MultiMedia January-Marchapply.2024
 PCB DEFECT DETECTION

6. Y.-J. Cha et al., “Autonomous structural visual no. 3, pp. 626–644, Mar. 2020, doi: 10.1109/TIM.2019.
inspection using region-based deep learning for 2963555.
detecting multiple damage types,” Comput.-Aided 19. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and
Civil Infrastructure Eng., vol. 33, no. 9, pp. 731–747, L.-C. Chen, “MobileNetV2: Inverted residuals and
Sep. 2018, doi: 10.1111/mice.12334. linear bottlenecks,” in Proc. IEEE Conf. Comput. Vis.
7. F. Yang, L. Huang, X. Tan, and Y. Yuan, “FasterNet- Pattern Recognit., 2018, pp. 4510–4520.
SSD: A small object detection method based on SSD 20. Y. Zhou, M. Yuan, J. Zhang, G. Ding, and S. Qin,
model,” Signal, Image Video Process., vol. 18, no. 1, “Review of vision-based defect detection research
pp. 173–180, 2024, doi: 10.1007/s11760-023-02726-5. and its perspectives for printed circuit board,”
8. T.-Y. Lin, P. Goyal, R. Girshick, and K. He, “Focal loss J. Manuf. Syst., vol. 70, pp. 557–578, Oct. 2023,
for dense object detection,” in Proc. IEEE Int. Conf. doi: 10.1016/j.jmsy.2023.08.019.
Comput. Vis., 2017, pp. 2980–2988.
9. J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation
XIN WANG is an associate professor with the School of
networks,” in Proc. Int. Conf. Comput. Vis. Image
Electrical and Control Engineering, Shenyang Jianzhu Univer-
Process., 2018, pp. 7132–7141.
sity, Shenyang, 110168, China. His research interests include
10. S. Woo, J. Park, J.-Y. Lee, I. Kweon, and C. So, “CBAM:
Convolutional block attention module,” in Proc. Eur. deep learning and defect detection. Wang received his
Conf. Comput. Vis. (ECCV), 2018, pp. 3–19. Ph.D. degree from Northeastern University. Contact him at
11. X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local [email protected].
neural networks,” in Proc. IEEE Conf. Comput. Vis.
Pattern Recognit., 2018, pp. 7794–7803. HONGYAN ZHANG is a master’s degree student at the
12. L. Zhu et al., “BiFormer: Vision transformer with School of Electrical and Control Engineering, Shenyang
bi-level routing attention,” in Proc. IEEE/CVF Jianzhu University, Shenyang, 110168, China. His research
Conf. Comput. Vis. Pattern Recognit., 2023, interests include deep learning and defect detection. Contact
pp. 10,323–10,333. him at [email protected].
13. Y.-J. Cha et al., “Deep learning-based crack damage
detection using convolutional neural networks,”
QIANHE LIU is a doctoral student at the School of Electrical
Comput.-Aided Civil Infrastructure Eng., vol. 32, no. 5,
and Control Engineering, Shenyang Jianzhu University, Shen-
pp. 361–378, 2017, doi: 10.1111/mice.12263.
yang, 110168, China. His research interests include deep
14. W. Choi and Y.-J. Cha, “SDDNet: Real-time crack
segmentation,” IEEE Trans. Ind. Electron., vol. 67, learning and defect detection. Contact him at 2238552865@
no. 9, pp. 8016–8025, Sep. 2020, doi: 10.1109/TIE.2019. qq.com.
2945265.
15. D. H. Kang and Y.-J. Cha, “Efficient attention-based WEI GONG is an associate professor with the School of Elec-
deep encoder and decoder for automatic crack trical and Control Engineering, Shenyang Jianzhu University,
segmentation,” Structural Health Monit., vol. 21, no. 5, Shenyang, 110168, China. His research interests include
pp. 2190–2205, 2022, doi: 10.1177/14759217211053776. defect detection. Gong received his master’s degree from
16. R. Ali and Y.-J. Cha, “Attention-based generative Jianzhu University. Contact him at [email protected].
adversarial network with internal damage
segmentation using thermography,” Automat. SONG BAI is an associate professor with the School of Elec-
Construction, vol. 141, Sep. 2022, Art. no. 104412,
trical and Control Engineering, Shenyang Jianzhu University,
doi: 10.1016/j.autcon.2022.104412.
Shenyang, 110168, China. His research interests include deep
17. J. Lewis, Y.-J. Cha, and J. Kim, “Dual
learning and defect detection. Bai received his master’s degree
encoder–decoder-based deep polyp segmentation
from Jianzhu University. Contact him at [email protected].
network for colonoscopy images,” Scientific Rep.,
vol. 13, no. 1, 2023, Art. no. 1183, doi: 10.1038/s41598-
023-28530-2. HEZHEN YOU is an undergraduate student at the Computer
18. Q. Luo, X. Fang, L. Liu, C. Yang, and Y. Sun, Science and Technology Department, Tongji University, Shang-
“Automated visual defect detection for flat steel hai, 200092, China. His research interests include defect detec-
surface: A survey,” IEEE Trans. Instrum. Meas., vol. 69, tion. Contact him at [email protected].

Authorized licensed
January-March 2024 use limited to: Zhejiang University. Downloaded on July 28,2024 at 15:05:37 UTC from IEEE
IEEE Xplore. Restrictions apply.
MultiMedia 87