DEEP LEARNING FOR

FACTORY AUTOMATION
Combining artificial intelligence with machine vision
 TABLE OF CONTENTS
What is deep learning?..................................... 3

Machine vision for assembly automation....... 4

The challenge of variability............................... 5

Advantages of human visual inspection......... 6

Deep learning for complex inspections.......... 7

Choosing between traditional

machine vision and deep learning................... 8

Cognex ViDi........................................................ 9

Conclusion.......................................................... 9

2 Introduction to Deep Learning

 DEEP LEARNING FOR FACTORY AUTOMATION
COMBINING ARTIFICIAL INTELLIGENCE WITH MACHINE VISION

WHAT IS DEEP LEARNING?

From the phones in our pockets to the reality of self-driving cars, the consumer economy has started to tap into
the power of deep learning’s neural networks. Deep learning has emerged as a foundational technology in the
speech, text, and facial recognition that we use in our mobile and wearable devices and is now beginning to be
used in many other applications—from medical diagnostics to Internet security—to predict patterns and make
critical business decisions. This same technology is now migrating into advanced manufacturing practices for
quality inspection and other judgment-based uses.
In essence, deep learning teaches robots and machines to do what comes naturally to humans: to learn by
example. New, low-cost hardware has made it practical to deploy bio-inspired, multi-layered “deep” neural
networks that mimic neuron networks in the human brain. This gives manufacturing technology amazing new
abilities to recognize images, distinguish trends, and make intelligent predictions and decisions. Starting from a
core logic developed during initial training, deep neural networks can continuously refine their performance as
they are presented with new images, speech, and text.
Deep learning-based image analysis combines the specificity and flexibility of human visual inspection with
the reliability, consistency, and speed of a computerized system. Deep learning models can precisely and
repetitively solve difficult vision applications that would be tedious to develop and frequently impossible to
maintain using traditional machine vision approaches. Deep learning models can distinguish unacceptable
defects while tolerating natural variations in complex patterns. And they can be readily adapted to new examples
without re-programming their core algorithms.
Deep learning-based software can now can perform judgment-based part location, inspection, classification,
and character recognition challenges more effectively than humans or traditional machine vision solutions.
Increasingly, leading manufacturers are turning to deep learning solutions and artificial intelligence to solve their
most sophisticated automation challenges.

Introduction to Deep Learning 3

 Human Inspectors
MACHINE VISION FOR
ASSEMBLY AUTOMATION
Gone are the days when humans directly
managed factory lines. Today, machines automate
manufacturing, assembly, and material handling
tasks. Machine vision systems equipped with
precision alignment and identification algorithms
and guidance capabilities have made it possible
to manufacture compact modern components that
could not be built manually. On a production line,
machine vision systems can inspect hundreds
Machine Vision
or thousands of parts per minute reliably and
+ Speed
repeatedly, far exceeding the inspection capabilities
of humans. + Accuracy
For decades, machine vision systems have taught + Repeatability
computers to perform inspections that detect
defects, contaminants, functional flaws, and other + Inspect details
irregularities in manufactured products. Machine too small to be
vision excels at the quantitative measurement of a seen by the
structured scene because of its speed, accuracy, human eye
and repeatability. A machine vision system built
around the right camera resolution and optics can Figure 1. Human inspectors are skilled at learning by example
easily inspect object details too small to be seen and appreciating acceptable deviations from the control. Machine
by the human eye, and inspect them with greater vision, by contrast, offers the speed and robustness that only a
reliability and less error (Figure 1). computerized system can.

Deep Learning Compared to Other Inspection Methods

Compared to Human Visual Compared to Traditional Machine

Inspection, Deep Learning is: Vision, Deep Learning is:

More consistent Designed for hard-to-solve applications

Operates 24x7 and maintains the same level of Solves complex inspection, classification and
quality on every line, every shift and every factory. location applications impossible or difficult with
classic rules-based algorithms.

More reliable Easier to configure

Identifies every defect outside of the set tolerance. Applications can be set up quickly, speeding up
proof of concept and development.

Faster Tolerates variations

Identifies defects in milliseconds, supporting high- Handles defect variations for applications that
speed applications and improving throughput. require an appreciation of acceptable deviations
from the control.

4 Introduction to Deep Learning

 THE CHALLENGE OF VARIABILITY
Traditional machine vision systems perform reliably
with consistent, well-manufactured parts. They
operate via step-by-step filtering and rule-based
algorithms that are more cost-effective than human
inspection. But algorithms become unwieldy as
exceptions and defect libraries grow. Certain
traditional machine vision inspections, such as final
assembly verification, are notoriously difficult to
program due to multiple variables that can be hard
for a machine to isolate such as lighting, changes in
color, curvature, and field of view (Figure 2).
Although machine vision systems tolerate some
variability in a part’s appearance due to scale,
rotation, and pose distortion, complex surface
textures and image quality issues introduce serious
inspection challenges. Machine vision systems
struggle to appreciate variability and deviation
between very visually similar parts (Figure 3).
Inherent differences or anomalies may or may
not be cause for rejection, depending on how the
user understands and classifies them. “Functional”
anomalies, which affect a part’s utility, are almost
always cause for rejection, while cosmetic anomalies
may not be, depending upon the manufacturer’s
needs and preference. Most problematically, these
defects are difficult for a traditional machine vision Figure 2. Application developers may struggle to program complex
system to distinguish between. inspections involving deviation and unpredictable defects into a
rules-based algorithm.

Wire present No wire

Figure 3. Confusing and glaring backgrounds can make it difficult for traditional machine vision systems to appreciate slight
differences between images. In this case, a deep learning-based model sees beyond the metal surface and specular glare to check
for missing wire bands in a car trim assembly.

Introduction to Deep Learning 5

 ADVANTAGES OF HUMAN INSPECTION
Unlike traditional machine vision, humans are adept at
distinguishing between subtle cosmetic and functional flaws,
as well appreciating variations in part appearance that may
affect perceived quality. Though limited in the rate at which
we can process information, humans are uniquely able to
conceptualize and generalize. We excel at learning by example
and are capable of distinguishing what really matters when it
comes to slight anomalies between parts. This makes human
vision the best choice, in many cases, for the qualitative Figure 4. Examples of complex scenes that human
interpretation of a complex, unstructured scene—especially vision excels at distinguishing.
those with subtle defects and unpredictable flaws (Figure 4).
For example, humans are much more accurate when dealing with deformed and otherwise hard-to-read
characters, complex surfaces, and cosmetic defects. For many of these applications, machines cannot compete
with humans for their appreciation of complexity.

DEEP LEARNING FOR COMPLEX INSPECTIONS

Deep learning models can help machines overcome their inherent limitations by marrying the self-learning of a
human inspector with the speed and consistency of a computerized system.
As the examples in Figure 5 show, deep learning-based image analysis is especially well-suited for cosmetic
surface inspections that are complex in nature: patterns that vary in subtle but tolerable ways, and where
position variants can preclude the use of methods based on spatial frequency. Deep learning excels at
addressing complex surface and cosmetic defects, like scratches and dents on parts that are turned, brushed, or
shiny. Whether used to locate, read, inspect, or classify features of interest, deep learning-based image analysis
differs from traditional machine vision in its ability to conceptualize and generalize a part's appearance based
upon its distinguishing characteristics—even when those characteristics subtly vary or sometimes deviate.

Figure 5. Deep learning-based image analysis excels at identifying cosmetic and functional anomalies that machine
vision struggles with, and it does so more quickly and reliably than a human inspector.

6 Introduction to Deep Learning

 CHOOSING BETWEEN TRADITIONAL MACHINE VISION AND DEEP LEARNING
The choice between traditional machine vision and deep learning depends upon the type of application being
solved, the amount of data being processed, and processing capabilities. Indeed, for its many benefits, deep
learning is not the right solution for many applications. Traditional rule-based programming technologies are
better at gauging and measuring, as well as performing precision alignment. In some cases, traditional vision
may be the best choice to fixture a region of interest precisely, and deep learning to inspect that region. The
result of a deep learning-based inspection may then be passed back to traditional vision to take accurate
measurements of the defect size and shape.
Deep learning complements rules-based approaches, and it reduces the need for deep vision domain expertise
to construct an effective inspection. Instead, deep learning has turned applications that previously required
vision expertise into engineering challenges solvable by non-vision experts. Deep learning transfers the logical
burden from an application developer, who develops and scripts a rules-based algorithm, to an engineer
training the system. It also opens a new range of possibilities to solve applications that have never been
attempted without a human inspector. In this way, deep learning makes machine vision easier to work with, while
expanding the limits of what a computer and camera can accurately inspect. Figure 6 below identifies the most
suitable applications for traditional machine vision and for deep learning-based approaches, including those
well-suited to either.

When to Deploy
Traditional Machine Vision vs. Deep Learning-Based Image Analysis

Gauging and measurement Complex cosmetic

inspection and segmentation

Inspection and
defect detection
Texture and material classification
Barcode reading and identification

OCR

Assembly verification
Presence/absence
Part and feature location

Deformed and variable feature location

Counting
Robotic guidance

Challenging OCR, including

distorted print

Figure 6. Deep learning-based image analysis and traditional machine vision are complementary technologies, with overlapping abilities as
well as distinct areas where each excels. Some applications may involve both technologies.

Introduction to Deep Learning 7

 COGNEX VIDI
Cognex ViDi™ is a ready-to-use deep learning-based technology dedicated to industrial image analysis.
Cognex ViDi is trained on labeled images that represent a part’s known features, anomalies, and classes—
much like a human inspector would be trained. A supervised training period teaches the system to recognize
explicit defects. For defects that come in multiple forms, the system trains itself in unsupervised mode to
learn the normal appearance of an object, including its significant but tolerable variations. Based on these
representative images, the software creates its reference model. It is an iterative process of constant
improvement, during which parameters can be adjusted and the outcome validated until the model works as
desired. During runtime, ViDi extracts data from a new set of images, and its neural networks localize parts,
extract anomalies, and classify them. Figure 7 explains the process of training and deploying a deep learning-
based application with Cognex ViDi.

TRAINING PHASE
1 Load sample images 2 Characterize 3 Train 4 Deliver results 5 Validate 6 Refine

Integrate to Machine

DEPLOYMENT PHASE
Acquire Analyze & 3 Deliver
1 images
2 interpret pass/fail result

PASS FAIL

Figure 7. Cognex ViDi allows technicians to train a deep learning-based model in minutes, based only on a small sample image set.
Once the application is configured, ViDi delivers fast, accurate results and saves images for process control.

Cognex ViDi works with a small set of training images, unlike the thousands typical of other deep learning
software. ViDi also works with limited computing power, requiring just one GPU card in a machine. Both
attributes make ViDi ideal for factory and manufacturing environments, where processing is PC-based and
image sets are limited. ViDi is field maintainable and can be retrained on the factory floor without a machine
builder or system integrator’s intervention. ViDi works with high resolution images, including color and thermal,
to recognize virtually any anomaly. It also performs complex counting and deciphers hard-to-read and deformed
characters. Tools for localization, characterization, classification, and OCR work independently or may be
combined with other Cognex vision tools to address complex vision challenges.
Cognex ViDi allows companies across many industries to create ground-breaking inspection systems that push
the boundaries of machine vision and invite the future of industrial automation. ViDi is available with VisionPro
and Cognex Designer machine vision software, giving customers a unique ability to mix and match tools within
an application.

8 Introduction to Deep Learning

 Human-like, powerful, and fast
Cognex ViDi combines the specificity and flexibility of human visual inspection with the reliability, repeatability,
and power of a computerized system—all in an easy-to-deploy interface.

ViDi Blue-Locate
finds complex
features and objects.

ViDi Red-Analyze
detects anomalies and
aesthetic defects.

ViDi Green-Classify
separates and classifies
objects or scenes.

ViDi Blue-Read
deciphers challenging
text and characters.

Figure 8. Cognex ViDi’s deep learning-based algorithms are optimized for real-world industrial image analysis, requiring vastly smaller
image sets and shorter training and validation periods. The Red-Analyze, Green-Classify, Blue-Locate, and Blue-Read tools solve surface
inspection, classification, location, and OCR applications.

CONCLUSION
Increasingly, industry is turning to deep learning technology to solve manufacturing inspections too complicated,
time-consuming, and costly to program using traditional rules-based algorithms. This will make it possible to
automate previously unprogrammable applications, reduce error rates, and quicken inspection times. Deep
learning offers manufacturers the possibility to solve problems which challenge traditional machine vision
applications, and to do so with greater robustness and reliability.

Introduction to Deep Learning 9

 BUILD YOUR VISION
2D VISION SYSTEMS
Cognex machine vision systems are unmatched in their ability to
inspect, identify, and guide parts. They are easy to deploy and provide
reliable, repeatable performance for the most challenging applications.

▪▪ Industrial grade with a library of advanced vision tools

▪▪ High speed image acquisition and processing
▪▪ Exceptional application and integration flexibility

www.cognex.com/machine-vision

3D LASER PROFILERS
Cognex In-Sight laser profilers and 3D vision systems provide ultimate
ease of use, power, and flexibility to achieve reliable and accurate
measurement results for the most challenging 3D applications.

▪▪ Factory calibrated sensors deliver fast scan rates

▪▪ Industry-leading vision software with powerful 2D and 3D tool sets
▪▪ Compact, IP65-rated design withstands harsh factory environments

www.cognex.com/3D-laser-profilers

IMAGE-BASED BARCODE READERS

Cognex industrial barcode readers and mobile terminals with
patented algorithms provide the highest read rates for 1-D, 2-D,
and DPM codes regardless of the barcode symbology, size, quality,
printing method, or surface.

▪▪ Reduce costs
▪▪ Increase throughput
▪▪ Control traceability

www.cognex.com/BarcodeReaders

Companies around the world rely on Cognex vision and barcode reading
solutions to optimize quality, drive down costs and control traceability.

Corporate Headquarters One Vision Drive Natick, MA 01760 USA

Regional Sales Offices

Americas Hungary +36 800 80291 Asia
North America +1 844-999-2469 © Copyright 2018, Cognex Corporation.
Ireland +44 121 29 65 163 China +86 21 6208 1133
Brazil +55 (11) 2626 7301 All information in this document is subject to change
Italy +39 02 3057 8196 India +9120 4014 7840
Mexico +01 800 733 4116 without notice. All Rights Reserved. Cognex is a
Netherlands +31 207 941 398 Japan +81 3 5977 5400
registered trademark of Cognex Corporation. ViDi
Poland +48 717 121 086 Korea +82 2 539 9980
Europe is a trademark of Cognex Corporation. All other
Spain +34 93 299 28 14 Malaysia +6019 916 5532
Austria +49 721 958 8052 trademarks are property of their respective owners.
Sweden +46 21 14 55 88 Singapore +65 632 55 700
Belgium +32 289 370 75 Lit. No. DLFAWP-05-2018
Switzerland +41 445 788 877 Taiwan +886 3 578 0060
France +33 1 7654 9318 Turkey +90 216 900 1696 Thailand +66 88 7978924
Germany +49 721 958 8052 United Kingdom +44 121 29 65 163 Vietnam +84 2444 583358 www.cognex.com