THE COMPANION GUIDE TO...
SMT INSPECTION
TODAY, TOMORROW, AND BEYOND
Improving Yield and Optimizing Processes with AI
LINKS TO TOPICS
› AI in the Factory of the Future
› Value of Accurate 3D Data
› Small Data, Big Challenges
› Minimize Human Resource by AI-based Autonomous Solution
› Measurement Getting Ever More Accurate
› Easy Changeover
› Improve Yield and Process Optimization
› KPO Screen Printer
› KPO Mounter
› Conclusion
AI in the Factory of the Future
Artificial Intelligence (AI) is a multidisci-
plinary field of science. The goal of AI is to
make machines “Smarter.” Historical appli-
cations of this goal include natural language
processing and translation, visual percep-
tion, pattern recognition, and decision making. The
number and complexity of applications have been
quickly growing across a variety of industries. Out
of all the advancements researchers have made, the current driver of the
AI inflection point is thanks to major advances in big data analysis by deep
learning. These advances have been limited exclusively to the human world
until now, but there are far-reaching applications within the manufacturing
sector, too. By using the right combination of AI technologies, manufacturers
can boost their efficiency, improve flexibility, speed up processes, and even
achieve self-optimizing processes at production sites. The SMT (surface mount
technology) industry is no exception. Currently facing a chronic shortage of
skilled labor, the electronics manufacturing industry can utilize and realize the
complete automation of the factory with the help of AI technologies. While
leading organizations such as IPC and SMTA (Surface Mount Technology Asso-
ciation) are trying to address the challenge with education programs and
training initiatives, it is still not enough due to the lack of consideration of
realities. Equipment providers like Koh Young are enabling the Smart Factory
of the Future by adopting AI to generate “knowledge” from “experience.”
On the other hand, machine-to-machine (M2M) communications, guided
in part by Industry 4.0, are quickly changing the manufacturing process by
aggregating process data such as first pass yield and throughput. Koh Young is
strengthening its large-scale datasets using the highest quality data captured
through its industry-leading 3D metrology expertise, then the AI algorithms
necessary for smarter manufacturing processes are trained by utilizing the
datasets. As part of this mission, the company is dedicated to adapting AI
technologies to inspection technologies that span a multitude of applications.
Indeed, Koh Young Technology is enabling the smart factory of the future by
adopting AI to provide Advanced Solutions as Smart Supporter, as shown in
Figure 1.
1
Figure 1: Advanced Solutions as Smart Supporter by Koh Young.
Value of Accurate 3D Data
Data is the most crucial element of the success of the AI solutions. Deep
learning effectiveness is linked to the quality and quantity of the input data
to address many different requirements from many fields. The use of AI to
provide smarter inspection systems has been desired by every inspection
provider. However, it has been difficult to realize due to the limitations of 2D
imaging, which was the de facto standard for the past 25 years. Not only is it
difficult for 2D Automated Optical Inspection (AOI) systems to identify defects
on curved and reflective solder joints, but 2D AOI systems cannot generate
reliable data. Every aspect of 2D inspection relies on 2D features like contrast;
thus, it is extremely challenging to correlate with the quantitative measure-
ment of 3D objects, and it is still a challenge to realize when the 3D measure-
ment data is unreliable.
Koh Young, however, measures the 3D information of the components and
solder joints. The Koh Young inspection machines (SPI and AOI) offer the most
valuable and reliable 3D measurement data in addition to 2D images, so they
become the most reliable “sensor” on the line (Figure 2). The validity of 3D data
is guaranteed as the company uses 3D technology for all component types to
extract the exact body dimensions, unlike other systems that use “blob detec-
tion,” “estimation,” or “interpolation,” which may be susceptible to external
factors like board warpage and component proximity. The combination of
measurement and process data piles collected from its SPI and AOI systems,
as well as from printers and mounters, allows Koh Young to deliver advanced
AI features with dependable “big data.” Indeed, the quality of data is more
important than the quantity required to create effective and reliable AI solu-
tions with high value proposition.
2
Figure 2: The inspection machines can be the “sensors.”
Small Data, Big Challenges
Unlike other industries where a model can be learned from a sufficiently
well-balanced and big enough dataset, the SMT manufacturing industry is
facing big challenges from having too little data. When AI-based solutions are
deployed in the real-life setting, we face such challenges as below:
1. Strict Objective: Reducing the number of false calls while not losing any of
the real No Good samples (NGs) has been a major challenge.
2. Imbalanced Dataset: In the modern production environment, most of the
data available for training are good samples (positive examples). It is difficult
to acquire real NG samples to provide a counter example for training the
model. In an extreme case, it is necessary to train the model using only good
or positive samples.
3. Catastrophic Forgetting: As with the problem above, we are observing a
new type of component day by day which is not in the training dataset. The
model needs to be trained and applied without losing previously learned
information.
4. Maintaining Training Dataset: It is difficult to afford the dataset if the
turnaround time or speed of the training is too long or slow. In the most ideal
situation, the new model should be trained with a fixed amount dataset, so
the training time and dataset can be manageable and finished within the
expected schedule.
5. Adaptable Solution Required: The integration of the AI solution should not
be a tight integration with the existing pipeline, so that it can be adapted
and applied for use in various other cases.
3
Over the many years, Koh Young has been working on various areas to address
these challenges by leveraging the world’s best analytical approaches for the
inspection and the state of art AI technologies.
Minimize Human Resource by AI-based Autonomous Solution
Koh Young is focused on developing AI applications for preparation, produc-
tion, optimization, and maintenance, while improving system performance, as
shown in Figure 3.
Figure 3: Koh Young AI-based Autonomous Solution.
From practical solutions like Koh Young Auto Programming (KAP) and Koh
Young Process Optimizer (KPO) to improvements in measurement quality and
inspection accuracy, Koh Young has been utilizing AI to meet the rising diffi-
culty in the SMT industry through its KSMART solution, as shown in Figure 4.
Figure 4: Koh Young KSMART Solution.
4
Measurement Getting Ever More Accurate
So, how does Koh Young use AI? It all begins with solving inspection chal-
lenges of SMT assemblies. The solder and components on finished boards
have many specular surfaces which will reflect some light back to the camera,
while creating strong inter-reflection with other lighting reflections. As some
of the reflected light does not reach the camera, they generate false signals,
which may cause measurement value errors. This specular reflection issue is
becoming increasingly troublesome in relation to increasing board density
and decreased component spacing. Koh Young uses AI to prevent measure-
ment errors by incorporating learning in the product. The hybrid fusion of
an analytic approach utilizing a mathematical measurement model and AI
used for learning abnormal symptoms from the combination of good and
bad measurement data allows Koh Young to detect and eliminate abnormal
measurements. This reduces false calls and escapes, as illustrated in Figure 5.
Through the hybrid fusion approach, the measurement accuracy only gets
better against many different challenges.
Figure 5: 3D measurement improvement with Koh Young AI engine.
Easy Changeover
Another area where Koh Young has been proactively applying AI is with
AOI programming. With the help of deep learning methods using true 3D
data, the assignment of components on a printed circuit board is gradually
becoming autonomous. The Koh Young AI-powered Auto Programming (KAP)
system proposes the recommended inspection conditions based on 3D-mea-
5
sured data, which not only simplifies inspection condition programming, but
also makes it faster and more consistent with the best mapping conditions.
Figure 6 describes the general process. KAP reduces job preparation by up
to 70 percent, which makes it an ideal solution for high-mix, low-volume, or
time-sensitive applications.
Figure 6: Koh Young Automatic Programming (KAP) solution.
Moreover, users can build up a User-KAP Library in addition to Koh Young-KAP
library for green site while verifying the result of KAP with accuracy rate (confi-
dence level) and origin of the mapped inspection conditions. Going further,
Koh Young will apply AI for automatic inspection condition tuning, while incre-
mentally learning new packages at new sites.
The most common issue that the industry is facing in the field is that the current
inspection pipeline judges NGs with strict criteria to not miss any real NGs.
While this helps avoid misclassification of true NGs, it results in many false
positives, like judging a component as an NG, even if correct. This requires
field engineers to manually verify the true NGs, which leads to additional
costs of time, money, and resources. Koh Young has been working on a Smart
6
Review solution to overcome this issue by a rule-based auto-classification and
incremental learning AI engine, which eventually gets transformed into the
Auto judgment system, as shown in Figure 7.
Figure 7: Roadmap of Koh Young Smart Review.
The rule-based auto-classification supports a reviewer’s decision by rule-based
auto classification with 3D measurement data, reduces unintended human
error (block defects), and controls the machine in real-time (configurable stop).
As shown in Figure 8, Koh Young uses 2D and 3D information intelligently to
automatically classify the defect type to minimize human intervention.
Figure 8: Overview of rule-based auto classification and block tolerance.
7
The Smart Review with incremental learning reevaluates the initial inspection
results at the review station by collecting operator’s feedback as a second level
filter. Due to this double-checking process, it can judge whether a component
is a false call or a true NG with high accuracy and low inference time, and with
minimal training dataset at the customer’s site.
An overview of the Smart Review process is shown in Figure 9.
Figure 9: An overview of the Smart Review with Incremental Learning.
Improve Yield and Process Optimization
In a surface mount assembly line, the setup and diagnosis of the printer and
mounter machines solely depend on human knowledge and expertise. Tens
of operators and engineers are needed to: 1) ensure a correct printer setup,
2) diagnose different printing problems (e.g., PCB support issue, squeegee
Figure 10: (a) KSMART Only; (b) Integrating KSMART and KPO.
8
blade defect), 3) initialize the printer machine with adequate parameters (e.g.,
printing speed, pressure, etc.), and 4) diagnose the mounter mechanical fail-
ures and analyze the root cause of the problems. KSMART supports some of
these tasks by collecting SPI and AOI data and conducting basic statistical anal-
ysis in real-time and serves as a monitoring tool. Koh Young Process Optimizer
(KPO), which uses AI algorithms to control and optimize printing and mounting
operations, as shown in Figure 10, is different from any other monitoring tool.
KPO is Koh Young’s smart factory solution driven by AI to control and opti-
mize the printing and mounting operations. KPO heavily relies on accurate
3D measurement data and error detection from SPI and AOI machines, which
sets the stage for smart factory solutions. Figure 11 outlines the KPO structure
and functionality by process type.
Figure 11: KPO smart factory solution.
9
KPO Screen Printer
The KPO printing solution includes three interlinking modules that exercise
complex algorithms to develop closed-loop print process recommenda-
tions, namely: Printer Diagnosis, Printer Advisor, and Printer Optimizer. The
enhanced AI engine actively optimizes the printing process by combining
real-time printing and SPI measurement data. Printer Advisor automati-
cally performs DOEs (design of experiments) devised to conduct a detailed
SPI result analysis using advanced diagnostic algorithms and noise filtering
models, and then recommends the ideal print parameters. Printer Diagnosis
uses multiple anomaly detection algorithms to actively optimize the print
process and further reduce false calls. The final module, called Printer Opti-
mizer, uses the Koh Young adaptive learning engine to generate models and
fine-tune process parameters. While each module provides inherent stand-
alone process benefits, the combined power of the three modules ensures
the highest process reliability and production flexibility while reducing dedi-
cated resources and expertise.
KPO Mounter
The KPO mounting solution includes three modules: Mounter Diagnosis,
Mounter Optimizer (Feedback), and Mounter Optimizer (Feedforward). KPO
Mounter Diagnosis module identifies the mounter internal issues through
placement offset patterns obtained from KPO AOI machines. It studies and
analyzes the offset distribution of the major mounter elements and differen-
tiates these offsets from the component offsets measured by the pre-reflow
AOI system. The system will automatically identify mechanical and software
mounter failures, as well as the root cause(s), while automatically notifying
technicians and engineers about mounter issues in real-time during produc-
tion. The main purpose of KPO Mounter Feedback module is allowing us
to zero out the average value of component placement offsets by utilizing
highly accurate measurement-based inspection data from Koh Young AOI
machines. The combination of the Mounter Diagnosis and Feedback modules
will provide an intelligent close loop solution which cannot adapt any new
change once the parameters have been fixed. On the other hand, the KPO
Mounter Optimizer (Feedforward) module proactively provides the optimal
offset values to the mounter for each component, which helps to minimize
defects after the reflow soldering process.
10
The ultimate step will be optimization of reflow oven by utilizing all measure-
ment data across SPI, pre-AOI, and post-AOI, so the entire manufacturing
process can be optimized from end to end starting from the volume of solder
to the temperature profile of the reflow oven. The KPO reflow oven solution
is currently being investigated and under development with our reflow oven
partners.
Conclusion
Artificial Intelligence and its associated benefits
will help the manufacturing industry confront
challenges, like the lack of skilled work-
force and rising costs. However, the
fundamental challenges caused
by small data problems, such as
imbalanced dataset, maintaining
training dataset, and catastrophic
forgetting, should be addressed to
meet the zero escape and flexible solution
requirement. Koh Young is focused on using an
AI-based solution as the primary vehicle to enable
the future of electronics manufacturing, while
addressing these issues with state-of-the-art
technologies and the best measurement data from our machines.
Technology and market leader Koh Young has set a long-term vision to
expand process capabilities and enhance factory performance by utilizing
our next-generation AI solutions, based on the core 3D measurement tech-
nology. To realize our vision, Koh Young continues to aggressively invest in
R&D and has established five additional R&D centers across the globe, collab-
orating universally to generate a quantum leap in technological leadership
and competitiveness. Koh Young’s expertise in AI solutions will pave the way
into new markets and go beyond the SMT industry.
11
12
About Koh Young Technology
Established in 2002, Koh Young revolutionized the inspection
market by launching the industry’s first 3D Solder Paste Inspection
(SPI) system using a patented dual-projection Moiré technology.
Since then, the company has become the global leader in 3D
measurement-based SPI and Automated Optical Inspection equip-
ment for the electronics industry. Based on its True3D™ measure-
ment-based inspection technology, Koh Young has developed
innovative inspection solutions for challenges with machined parts,
press-fit and through-hole pins, conformal coatings, dispensed
materials, and semiconductor packages. Through its constant
innovation, Koh Young has secured over 3,500 global customers,
and commands the dominant global market share position in the
SPI and AOI markets with well over 20,000 machine installations.
By adopting a user-centric R&D focus, the company continues to
use its core competencies to develop innovative solutions for new
and existing markets by listening to our customers and researching
emerging trends and applications. With headquarters in Korea,
Koh Young has a global sales and support infrastructure spanning
Europe, Asia, and the Americas. These regional offices ensure Koh
Young stays close to the market and, more importantly, its growing
user base. You can learn why so many electronics manufacturers
trust Koh Young for reliable inspection solutions at our regional
website kohyoungamerica.com.
13
