Machine Vision Hassndbook
Machine Vision Hassndbook
Machine Vision
Handbook
With 1295 Figures and 117 Tables
Editor
Bruce G. Batchelor
Professor Emeritus
School of Computer Science and Informatics
Cardiff University
Cardiff, Wales
UK
Please note that additional material for this book can be downloaded from
http://extras.springer.com
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted
under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or
transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case
of reprographic reproduction in accordance with the terms of licenses issued by the Copyright Licensing Agency.
Enquiries concerning reproduction outside those terms should be sent to the publishers.
The use of registered names, trademarks, etc., in this publication does not imply, even in the absence of a specific
statement, that such names are exempt from the relevant laws and regulations and therefore free for general use.
The publisher makes no representation, express or implied, with regard to the accuracy of the information
contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may
be made.
For those I hold dearest: my wife Eleanor, mother Ingrid, daughter Helen,
son David and my seven grandchildren.
Preface
When people ask me what work I do, I say that I am an academic engineer, studying Machine
Vision and therefore call myself a Machine Vision Systems Engineer (Vision Engineer for short).
When asked to explain what that means, I reply in the following way:
‘‘I study machines that can ‘see’. I use a video camera, connected to a computer, to find defects
in industrial artifacts as they are being made.’’
In my experience, this is just short enough to avoid total boredom in fellow guests at dinner
parties. However, it is woefully inadequate, since it does not encompass many topics that are
essential for building a successful Machine Vision system. Compare the ‘‘dinner-party defini-
tion’’ with the following, more formal statement, which introduces the broader concept of
Artificial Vision:
Artificial Vision is concerned with the analysis and design of opto-mechanical-electronic
systems that can sense their environment by detecting spatio-temporal patterns of electro-magnetic
radiation and then process that data, in order to perform useful practical functions.
Machine Vision is the name given to the engineering of Artificial Vision systems. When
browsing through the technical literature, the reader will soon encounter another term:
Computer Vision (CV) and will quickly realise that it and Machine Vision (MV) are often
used synonymously. This is a point on which I strongly disagree! Indeed, this book is founded
on my firm belief that these subjects are fundamentally different. There is a certain amount of
academic support for this view, although it must be admitted that the loudest voices of dissent
come from within universities. On the other hand, most designers of industrial vision systems
implicitly acknowledge this dichotomy, often by simply ignoring much of the academic
research in CV, on the grounds that it is not relevant to their immediate needs. In the ensuing
pages, I and my co-authors argue that Machine Vision should be recognised as a distinct
academic and practical subject that is fundamentally different from Computer Vision.
Computer Vision, Artificial Intelligence, Pattern Recognition and Digital Image Processing
(DIP) all contribute to MV, which makes use of algorithmic and heuristic techniques that were
first devised through research in these other fields. Machine Vision concentrates on making
them operate in a useful and practical way. This means that we have to consider all aspects
of the system, not just techniques for representing, storing and processing images inside a
computer. With this in mind, we come to the following working definition.
Machine Vision is concerned with the engineering of integrated mechanical-optical-electronic-
software systems for examining natural objects and materials, human artifacts and manufacturing
processes, in order to detect defects and improve quality, operating efficiency and the safety of both
products and processes. It is also used to control machines used in manufacturing. Machine Vision
requires the harmonious integration of elements of the following areas of study
● Mechanical handling
● Lighting
● Optics (conventional, fibre optics, lasers, diffractive optics)
● Sensors (video cameras, UV, IR and X-ray sensors, laser scanners)
● Electronics (digital, analogue and video)
viii Preface
● Signal processing
● Image processing
● Digital systems architecture
● Software
● Industrial engineering
● Human-computer interfacing
● Control systems
● Manufacturing
● Existing work practices and QA methods.
Machine Vision has also been applied successfully to several high-volume niche markets,
including:
● Reading automobile registration plates
● Face recognition (security purposes)
● Fingerprint recognition
● Iris recognition
● Document processing
● Signature verification
● Security surveillance
● Print inspection
● Fabricating micro-electronic devices
● Karyotyping (chromosome identification and classification)
● Inspecting bare printed cicuit boards
● Controlling/checking the placement of components on printed circuit boards.
However, we shall not discuss these niche application areas, preferring instead to concen-
trate on manufacturing industry, which is distinctive in presenting many millions of low-volume
potential applications. These vary from trivial to extremely difficult. Rather than solving one
application at a time, I have spent my working life devising tools that are appropriate for a very
wide range of situations. The aim of my work over the last 35 years has been to build versatile
development systems that can be used in the study of newly defined industrial inspection,
measurement or control tasks. Using such systems enables new applications to be solved, or
dismissed as impractical, in a very short time. There is a never-ending stream of new industrial
applications. In the 1980s, the 3M Company, with whom I was privileged to work, introduced
more than 10 new products each day! The challenge we faced was to devise tools allowing the
potential benefits of MV to be explored quickly and cheaply; we could spend no more than
a few hours evaluating each new application. Contrast this with the niche markets mentioned
above, where there is the potential for a high financial return for a single design. Unlike most
industrial MV projects, these niche markets were able to justify the high design costs needed.
Let us be more specific about what we include within the scope of our discussion in this
book. We concentrate on manufacturing industry: ‘‘metal bashing’’ and processing materials
such as plastics, wood, glass, rubber, and ceramics. A similar approach is appropriate in the
manufacture and packaging of food, beverages, pharmaceuticals, toiletries, clothing and
furniture. Many of the same ideas are also applicable to processing fabrics, leather, minerals,
plant materials, fruit, vegetables, nuts, timber, animal and bird carcasses, meat, fish, etc.
The problem of nomenclature arises because MV, CV and DIP are all concerned with the
processing and analysis of pictures within electronic equipment. I did not specify computers
Preface ix
explicitly, because Artificial Vision does not necessarily involve a device that is recognizable as
a computer. Of course, it is implicit in the very name of CV that image processing takes place
inside a computer. On the other hand, MV does not impose such a limitation. For example,
it also allows the processing of images to take place in specialised electronic or electro-
optical hardware.
Although MV, CV and DIP share a great many terms, concepts and algorithmic techniques,
they require a completely different set of priorities, attitudes and mental skills. The division
between MV and CV reflects the division between engineering and science. Engineers have long
struggled to establish a clear distinction between engineering and science. This book tries to
do the same for MV Systems Engineering and its scientific-mathematical counterpart: Com-
puter Vision.
CV is usually concerned with general questions, such as ‘‘What is this?’’. On the other hand,
MV almost always addresses very specific questions, for example ‘‘Is this a well made widget?’’.
This reflects one very simple fact: in a factory, we know what product is being made at any given
moment, so we know what an object should look like. As a result, MV systems are usually used
for verification, not recognition. The reverse is often true for CV. As we shall see in the following
pages, this has profound implications.
The people responsible for generating MV data (i.e. images) are usually cooperative.
A manufacturing engineer wants to produce a good product and will, as far as is practical,
modify the inspection millieu, or even the product itself, to improve matters. Contrast this with
the task of designing systems for recognising military targets. The enemy does not want to
cooperate and actually tries to obscure his presence and actions! Similar remarks apply to
fingerprint identification and forensic image analysis. Not surprisingly, criminals show little
interest in cooperating with those people who are trying to catch them! In medicine, the
situation is little better; tumours do not cooperate with equipment designers, although
a battery of techniques (e.g. staining) has been developed for highlighting the presence of
cancer cells. On a continuum scale recording the degree of cooperation that we can expect, MV
applications are clustered at one end and CV applications near the other.
In MV, the allowed unit cost for analysing each image is likely to be very low. In CV
applications, such as military target recognition and oncology, the cost of making a mistake can
be very high indeed, so a higher unit cost per scene inspected can often be tolerated. Most CV
problems are ‘‘difficult’’ in terms of artificial intelligence, while MV is usually ‘‘easy’’.
The distinction between MV and CV can be illustrated by considering a seemingly simple
task: determining whether a coin has its obverse, or reverse, face uppermost. (This might well
be a component in the bigger process of inspecting proof-quality coins.) CV begins with an
image that was obtained by somebody else, not the CV specialist. On the other hand, MV
begins with an embossed metal disc. Deriving an image from it requires that the MV engineer
designs the lighting, optics and mechanical handling of the coins, as well as choosing an
appropriate camera. The crucial importance of lighting and optics is obvious from the fact that
very different images of the same coin can be produced, simply by moving the lights very
slightly. In the MV engineer’s mind, consideration of the representation, storage, processing
and analysis of images follows once a good image has been produced. Computer processing
of images for MV is important but it is not the vision engineer’s only concern. He cannot afford
to concentrate on this, or any other part of a system, and then neglect all others. This is the
primary and inviolate rule for Machine Vision system design.
During the 1970s and 1980s, Japan taught the rest of the world the importance of ensuring
the highest quality in manufactured goods. The West learned the hard way: markets were lost to
x Preface
companies whose names were hitherto unknown. Many long-established and well-respected
companies were unable to meet the challenge and failed to survive. Those that did were often
faced with difficult years, as their share of the market shrank. Most companies in Europe
and America have come to terms with this now and realize that quality has a vital role in
establishing and maintaining customer loyalty. Hence, any technology that improves or simply
guarantees product quality is welcome. Machine Vision has much to offer manufacturing
industry in improving product quality and safety, as well as enhancing process efficiency and
operational safety. Machine Vision owes its rising popularity to the fact that optical sensing is
inherently clean, safe, and versatile. It is possible to do certain things using vision that no other
known sensing method can achieve. Imagine trying to detect stains, rust or surface corrosion
over a large area by any means other than vision!
The recent growth of interest in industrial applications of Machine Vision is due, in large
part, to the falling cost of computing power. This has led to a proliferation of vision products
and industrial installations. It has also enabled the development of cheaper and faster com-
puting machines, with increased processing power. In many manufacturing companies, serious
consideration is being given now to applying Machine Vision to such tasks as inspecting
grading, sorting, counting, monitoring, measuring, gauging, controlling and guiding. Auto-
mated Visual Inspection systems allow manufacturers to keep control of product quality, thus
maintaining / enhancing their competitive position. Machine Vision has also been used to
ensure greater safety and reliability of the manufacturing processes. The whole area of flexible
automation is an important and growing one, and Machine Vision will continue to be an
essential element in future Flexible Manufacturing Systems. There are numerous situations
where human inspection is unable to meet production demands. The advent of Machine
Vision has often enabled the development of entirely new products and processes. Many
companies now realise that Machine Vision forms an integral and necessary part of their
long-term plans for automation. This, combined with the legal implications of selling defective
and dangerous products, highlights the case for using Machine Vision in automated inspec-
tion. A similar argument applies to the application of vision to robotics and automated
assembly, where human operators were previously exposed to dangerous, unhealthy / or
unpleasant working conditions. Machine Vision is a valuable tool in helping to avoid this.
No Machine Vision system existing today, or among those planned for the foreseeable
future, approaches the interpretative powers of a human being. However, current Machine
Vision systems are better than people at some repetitive quantitative tasks, such as making
measurements under tightly controlled conditions. Machine Vision systems can out-perform
people, in certain limited circumstances. Industrial vision systems can generally inspect simple,
well-defined mass-produced products at very high speeds, whereas people have considerable
difficulty making consistent inspection judgments in these circumstances. Machine Vision
systems exist that can inspect several thousand objects per minute, which is well beyond human
ability. Studies have shown that, at best, a human inspector can only expect to be 70–80%
efficient, even under ideal conditions. On many routine inspection tasks, a Machine Vision
system can improve efficiency substantially, compared to a human inspector. A machine can,
theoretically, do this for 24 hours / day, 365 days / year. Machine Vision can also be useful in
detecting gradual changes in continuous processes. Tracking gradual colour, shade or texture
variations is not easy for a person.
Currently, the main application areas for industrial vision systems occur in automated
inspection and measurement and, to a lesser extent, robot vision. Automated Visual Inspection
and measurement devices have, in the past, tended to develop in advance of robot vision
Preface xi
systems. In fact, QC-related applications, such as inspection, gauging and verfication, currently
account for well over half of the industrial Machine Vision market. This has been achieved, in
many cases, by retrofitting inspection systems onto existing production lines. There is a large
capital investment involved in developing a completely new robotic work cell. Moreover, the
extra uncertainty and risks involved in integrating two relatively new and complex technologies
makes the development of robot vision system seem a daunting task for many companies and
development has often lagged behind that of inspection devices. The technical difficulties
involved in controlling flexible visually guided robots have also limited the development.
On the other hand, Automated Visual Inspection systems now appear in every major industrial
sector, including such areas as consumer goods, electronics, automobile, aerospace, food,
mining, agriculture, pharmaceuticals, etc.
Machine Vision systems for industry first received serious attention in the mid-1970s.
Throughout the early 1980s, the subject developed slowly, with a steady contribution being
made by the academic research community, but with only limited industrial interest being
shown. It seemed in the mid-1980s that there would be a major boost to progress, with serious
interest being shown in vision systems by the major American automobile manufacturers.
Then, came a period of disillusionment in the USA, with a large number of small vision
companies failing to survive. In the late 1980s and early 1990s, interest grew again, due largely
to significant progress being made in making fast, dedicated image processing hardware.
During the first decade of the 21st century, the growth of the Machine Vision market was
unrelenting. This was helped considerably by the development of new technologies, especially
plastic optical components, LED lighting, high-resolution cameras and fast computer hard-
ware. For many applications, it is possible now to provide sufficiently fast processing speed on
a standard computing platform. Throughout the last 35 years, academic workers have dem-
onstrated feasibility in a very wide range of products, representing all of the major branches of
manufacturing industry. I have been very privileged to have participated in this.
I have watched this subject develop and blossom, from the first tentative steps taken in the
mid-1970s to today’s confident multi-billion dollar industry. In those early days, industrial
engineers and managers were mentally unprepared for the new technology. Although there is
far greater acceptance of vision system technology nowadays, many customers are still unwill-
ing to accept that installing and operating a vision system places a new intellectual respon-
sibility on them. They often fail to understand that machines and people see things very
differently. During the last 35 years, I have seen similar mistakes being made, through a lack of
appreciation of this point. Sadly, customer education is still lagging far behind technological
developments in Machine Vision. The responsibility for making Machine Vision systems work
properly does not lie solely on the shoulders of the vision engineer. The customer has a duty
too: to listen, read, and observe, Making the most from the strengths and weaknesses of
Machine Vision technology involves appreciating a lot of subtle points. This book discusses
just a small proportion of these. My colleagues and I have had great fun developing and
writing about this amazing technology and hope that some of our excitement will be evident in
these pages.
It is customary in many areas of writing to use so-called gender-neutral constructions, such
as ‘‘he/she’’, ‘‘s/he’’, ‘‘his/her’’ etc. I regard these as being both clumsy and counter-productive.
To improve clarity and to avoid placing women after men, I use the words, ‘‘he’’, ‘‘his’’ and
‘‘him’’ in the traditional way, to encompass both sexes, without granting precedence to either.
I am indebted to my friends and colleagues who have contributed to this handbook.
Without exception, their enormous experience is evident in the splendid essays that grace
xii Preface
this description of this exciting subject. I was inspired to study Machine Vision systems when
I was preparing a book (Pattern Recognition: Ideas in Practice, Plenum, New York & London,
1978, ISBN 0-306-31020-1) with invited contributions from expert authors. One of them,
Dr. John R Parks, offered a chapter entitled Industrial Sensory Devices, which so excited me that
37 years later I am still conducting research in that area. It is a pleasure to acknowledge John
Parks, for his foresight, technical knowledge and humour. This book describes an interactive
image processing system, called QT, which has its origins, in part, to work in the 1970s on
SUSIE (Southampton System for Image Evaluation) by Dr. Jonathan Brumfitt. Toby Holland
developed the video interface hardware for SUSIE, which inspired the development of the
AutoView Viking system. (British Robotic Systems Ltd.) Peter Heywood, Dr. Graham Page,
Dr. David Mott and David Upcott all contributed to that. In parallel, Tim Loker, Mike
Rippingale and Martin Reeves, of Vision Dynamics Ltd, developed a similar system called
VCS. The Intelligent Camera (Image Inspection Ltd.) also followed from SUSIE and was
developed by Dr. Piers Plummer. Dr. David Mott integrated AutoView Viking with Prolog.
Inspired by this work, VCS, the Intelligent Camera and QT (described in detail here) were,
in turn, interfaced to Prolog. Dr. Andrew Jones, Ralf Hack and Stephen Palmer wrote an
integrated package, called PIP, incorporating LPA Prolog (Logic Programming Associates Ltd)
and our own image processing software, written in C. I built QT, on top of the MATLAB™
Image Processing Toolbox (Mathworks, Inc). Recently, QT has itself been extended by Dr. Simon
Caton, who developed a multi-host, multi-process version. Dr. Luke Chatburn developed a
web-based, multi-computer system, called Myriad, for image processing. Dr. Frederick Waltz
and I worked together for many years, applying interactive image processing to real-world
industrial inspection, measurement and control tasks. Michael Daley, Garry Jackson and
Richard Crowther all worked with me, interfacing image processing systems to various
electro-mechanical devices, including our model train. In addition, Dr. Barry Marlow,
Dr. Simon Cotter, Dr. Chris Bowman, Dr. Tony McCollum, Dr. John Chan, Dr. Mark Graves,
Dr. Graham Wilson, Dr. Robin Tillet, Prof. Paul Whelan, and Rolf Iverson all helped to guide
my thinking. I am pleased to express my gratitude to all of the former colleagues that I have
mentioned above. I recall with great pleasure the fulfilling times that I have enjoyed working
with them all. It was great fun! My special thanks are due to Tim Loker, Peter Heywood and
Fred Waltz who opened doors for me.
This book would not have been completed without the unerringly patient support and
encouragement of my dear wife Eleanor. Gratitude is not enough for her love!
My clumsy attempts at building machines that can ‘‘see’’ simply emphasizes how wonderful
and mysterious my own sense of sight is. It is humbling to realise that our supposedly
sophisticated technology is no match for the gift of sight that our Creator graciously chose to
bestow on us. It is well to remember that we can at best supplement but not replace
human vision.
Bruce G. Batchelor
Cardiff, Wales, United Kingdom
November 2011
Table of Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
List of Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
Volume 1
10 Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
David L. Gilblom
Volume 2
Volume 3
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2209
List of Contributors