Proceedings of the 2000 IEEE

International Conference on Robotics 8 Automation

San Francisco, CA April 2000

ICRA 2000 Symposium on

NEW TRENDS IN PROTOTYPING DESIGN AND AUTOMATION
Kok-Meng Lee
George W. Woodruff School of Mechanical Engineering
Georgia Institute of Technology
Atlanta, GA 30332-0405
Tel: (404)894-7402; Fax: (404)894-9342
email: [email protected]

Tarek M. Sobh,
School of Engineering and Design
University of Bridgeport
169 University Avenue Bridgeport, CT 0660 1, U.S.A.
Tel: (203)576-4116; Fax: (203)576-4766
e-mail: [email protected]

Abstract
Recent developments in several other emerging fields
enable us to think of the fields of prototype design, development
and automation in new ways and to consider new applications.
Most notably are the emerging fields on Very Large Scale
Integration (VLSI), modern bio-medical engineering, and the
world-wide-web Internet communication technology. For these
reasons, we decided at the outset of this symposium to focus on
the use of information technology for prototype design,
development and automation, and yet to cover a wide variety of
novel applications with emphasis on emerging fields of
automation. Based on the above principles, we organized this
symposium which contains five papers, covering the topics of
information extraction for engineering design, predictive models
of discrete-event and hybrid systems, process planning
automation, and prototyping design of vision sensing systems,
microhano systems, and automated DNA sequencing. This
overview paper presents trends of prototyping design and
automation with an emphasis on the following two subtopics: (1)
a review on prototyping discrete event and hybrid systems, and
(2) the trends on prototyping real-time machine vision system
design.
1. INTRODUCTION

During the last two decades, the rapid advancement of

computer, communication, and control technologies have greatly
accelerated the efforts of developing novel prototypes and their
cost-effective applications in automation. Today, besides
introducing the intelligence directly into equipmentshystems
through embedded microcomputers and providing virtual
prototyping through enhanced CAD/CAE facilities, information
flow has been well regarded as an essential part of the integrated
design approach whereby all members of the prototype
development and manufacturing automation team can work
closely together throughout the design and manufacturingcycle.
In parallel to the above advancements, recent development
in several other emerging fields enabled us to think of the fields
of prototype development and automation in new ways and to
consider new applications. Perhaps, the three most notable
emerging fields are the VLSI, modem bio-medical engineering,
and the world-wide-web Internet communication technology.
For these reasons, we decided at the outset of this symposium to
focus on the use of information for prototype design,
development and automation, and yet to cover a wide variety of
novel applications with emphasis on emerging fields of
automation. There are two broadly defined types of information
essential to ensure the optimum performance of automation;
namely, off-line knowledge databases and predictive models,

0-7803-5886-4/00/$10.00@2000 IEEE

184

and on-line sensing (or real-time feedback). The former

provides a baseline of "in-advance'' information but if routine
deviations are greater than can be tolerated, the latter is needed
to augment this baseline information for feedback to controllers.
Based on the above principles, this symposium contains five
papers, covering the topics of information extraction for
engineering design, predictive models of discrete-event and
hybrid systems, process planning automation, and prototyping
design of vision sensing systems, microhano systems, and
automated DNA sequencing. The symposium is organized as
follows. Following this overview paper, Fukuda and Arai will
deliver the second paper, which addresses the recent research
results and issues in prototyping design, control and automation
of miniature manipulation systems considering the dominant
physical phenomena in the microhano world. Another novel
prototyping design, offered by Meldrum in the third paper,
presents the prototyping of an automated fluid sample handling
system, which has potential applications to significantly reduce
the time of DNA sequencing, diagnostics, forensics, protein
crystallography, and so on. The fourth paper by Kusiak
introduces the concept of data mining for extracting meaningful
features (attributes) from large databases and makes accurate
predictions in industrial, business, and medical applications.
Kusiak's paper attempts to address interesting challenges posed
by the growing volume of information. In the last paper,
Riesenfeld et al. describe a method for automatically generating
a process plan from a high-level shape feature part description.
In demonstrating the concept, their system is targeted at a
restricted, but broadly useful, class of machined metal or plastic
prismatic parts with four and five axis indexing.
The remainder of this paper will give an overview on trends
of prototyping design and automation with an emphasis on the
following two subtopics. The first is an overview for the
development of a theory for prototyping discrete event and
hybrid systems. Discrete Event Dynamic Systems (DEDS) are
dynamic systems in which state transitions are caused by
internal, discrete events in the system. DEDS are attracting
considerable interest and current applications are found in
manufacturing systems, communications and air traffic systems,
robotics, autonomous systems, and artificial intelligence. We
also present an overview for the development of a simple
graphical environment for simulating, analyzing, synthesizing,
monitoring, and controlling discrete event and hybrid systems.
The second subtopic is an overview of machine vision
prototyping design for real-time automation applications. We
discuss the problems associated with traditional machine vision
systems for cost-effective real-time applications, novel
alternative system design to overcome these problems, and the

new trends of modern vision sensors. Modern smart sensors

provide the features of a traditional machine vision system at
less than half of the usual price by eliminating the signalconversion electronics, fixed-frame rates and limited gray-scale
quantization. Camera, image-acquisition electronics, and
computer are integrated into a single unit to allow dynamic
access to the CCD without image float or flutter. We also
present a physically-accurate image synthesis method as a
flexible, practical tool for examining a large number of
hardwarehoftware configuration combinations for a wide range
of parts.
2. DISCRETE EVENT AND HYDRID SYSTEMS
The underlying mathematical representation of complex
computer-controlledsystems is still insufficient to create a set of
models which accurately capture the dynamics of the systems
over the entire range of system operation. We remain in a
situation where we must trade off the accuracy of our models
with the manageability of the models. Closed-form solutions of
mathematical models are almost exclusively limited to linear
system models. Computer simulation of nonlinear and discreteevent models provide a means for off-line design of control
systems. Guarantees of system performance are limited to those
regions where robustness conditions apply. These conditions
may not apply during startup and shutdown or during periods of
anomalous operation.
Recently, attempts have been made to model low and highlevel system changes in automated and semi-automatic systems
as discrete event dynamic systems (DEDS). Several attempts to
improve the modeling capabilities are focused on mapping the
continuous world into a discrete one. However, repeated results
are available which indicate that large interactive systems evolve
into states where minor events can lead to a catastrophe. Discrete
event systems (DES) have been used in many domains to model
and control system state changes within a process. Some of the
domains include the following: Manufacturing, Robotics,
Autonomous Agent Modeling, Control Theory, Assembly and
Planning, Concurrency Control, Distributed Systems,
Hierarchical Control, Highway Traffic Control, Autonomous
Observation Under Uncertainty, Operating Systems,
Communication Protocols, Real-Time Systems, Scheduling, and
Simulation.
A number of tools and modeling techniques are being used
to model and control discrete event systems in the above
domains. Some of the modeling strategies include: Timed,
untimed and stochastic Petri Nets and State Automata,
Markovian, Stochastic, and Perturbation models, State
Machines, Hierarchical State Machines, Hybrid Systems
Modeling, Probabilistic Modeling (Uncertainty Recovery and
Representation), Queuing Theory, and Recursive Functions.
The focus of Sections 2 and 3 will be to present a brief
review for prototyping discrete event and hybrid systems,
discuss some techniques used in the DEDS field, and present a
simple software prototyping tool for representing hybrid DES.

representing hybrid systems which contain one or more of the

following characteristics:
0
continuous domain,
discrete domain,
0
chaotic behavior, and
0
symbolic parameters.
Some examples of DEDS are
Data Network A ={send, receive, timeout, lost}
Shop with k jobs: A={admitjob,job-finished}
Electric Distribution:A= {normal,short-circuit, over-current}
There are several frameworks that can be used to model
DEDS such as: finite automata, Petri nets, Markov chains, etc.
Choosing one of these frameworks depends on the nature of the
problem being modeled and the implementation techniques
available to implement this model.
DEDS has been applied to model many real-time problems
and has been involved in different types of applications. Some of
these applications are:
0
Networks
0
Manufacturing (sensing, inspection, and assembly)
0
Economy
0
Robotics (cooperating agents)
Highway traffic control
0
Operating systems
For more details about DEDS applications see [ 1, 2,4, 5 , 61. We
believe that DEDS will have an important role in the
development and improvement of many other applications in
different disciplines.
2.2 Discrete Event Models
As mentioned before, there are several representations and
frameworks used in DEDS modeling. Some of these frameworks
are:
0
Automata (untimed, timed, temporal, stochastic).
0
Pushdown automata, precursive, and Turing machines.
0
Petri nets (timed, untimed).
0
Markov chains.
0
Queuing theory.
0
Min-Max Algebra.
0
Uncertainty modeling.
0
Classical control.
These frameworks can be categorized in three different domains:
Timed vs. untimed models: the untimed models emphasize the
"state-event sequence" of a DEDS and ignore the holding
time of each state, while in the timed models, "time" is an
essential part of the model.

2.1 Hybrid and Discrete Event Systems

Discrete event dynamic systems (DEDS) are dynamic
systems in which state transitions are triggered by the occurrence
of discrete events in the system. DEDS are suitable for

185

Deterministic vs. probabilistic models: deterministic models

assume pre-knowledge of the sequence of events that will
occur at any time, while probabilistic (stochastic) models
associate probabilities with each event.
Computational model: which can be logical models in which
the primary questions are of qualitative or logical nature,
while algebraic models can capture the description of the
trajectories in terms of a finite set of algebraic operations.
Finally, performance models are formed in terms of
continuous variables such as average throughput, waiting
time, etc.

Logical

Algebraic
4

1
1

Performance

1
I

Timed
Temporal Logic
Timed Petri Nets

Untimed
Finite State Machines
Petri Nets

Min-Max Algebra

Finitely Recurcive
Proc. CO".
Sequencial proc.

computation time), probabilistic transitions, controllability and

observability definitions, temporal, timed, state space, Petri-nets,
and recursive representations, analysis, and synthesis algorithms.
The environment allows not only the graphical construction and
mathematical analysis of various timing paths and control
structures, but also produces C code to be used as a controller for
the system under consideration.
Using the environment is fairly simple. For finite state
machines the designer uses the mouse to place states
(represented by ovals) and connect them with events
(represented by arrows). Transitions and states can be added,
moved and deleted easily. Figure 2 is an example of a simple
stochastically timed FSM, containing 4 states and 5 events.

Markov Chains
Queueing Networks
GSMF'?Simulation
Stochastic Petri Nets
Stochastic+

I
tNonstochastic

Language complexity is based on :.t formal theory of languages.

Each FSM generates a language L which represents all
possible traces of this FSM.
L(FSM) cL(Petrinets)
So, Petri nets is more language complex than FSM.
Algebraic complexity is based on the systems theory. We can
consider any algebraic system as a set of models and a set of
operators that map one or more model to another. For
example, in transfer functions, addition and multiplication
reflect serial and parallel systems.
Logical correctness is a desirable property of the traces
generated by any DEDS model. For example, in the data
network example, we must guarantee that each transmitted
packet has been received correctly by the receiver.
Real-time requirement is a desirable property of the real-time
response of the actual system. It is necessary to embed the
DEDS model in a real-time environment.

3. A SIMPLE PROTOTYPING TOOL

We have built a software environment to aid in the design,
analysis and simulation of Discrete Event and Hybrid Systems.
The environment allows the user to build a system using either
Finite State Machines or Petri-Nets. The environment runs
under m o t i f and supports a graphical DES (Discrete Event
System) hybrid controller, simulator, and analysis framework.
The framework allows for the control, simulation and
monitoring of dynamic systems that exhibit a combination of
symbolic, continuous, discrete, and chaotic behaviors, and
include stochastic timing descriDtions (for events, states, and

186

Figure 2 A Stochastically timed FSM window during analysis

The probabilities on the events (that is, which path to navigate in
the automaton) are designated using the mark field in the status
dialog box. The different timings (on event and state times) and
distribution function type, mean and variance can be assigned
through the status dialog box too. The allowable distributions are
currently restricted to Gaussian and exponential functions, but
can be easily extended to arbitrary discrete or continuous
distributions. A window shows the distribution function at a
state or event, and also allows the user to do queries. For
example: queries on whether a path time probability is greater or
less than a given time, or combined timing distributions to reach
a goal state through various paths, etc. The dialog box allows
the user to perform queries of various kinds. The currently
selected state/event is drawn with a dashed line, and the
information in the status window pertains to it. Optimizing paths
based on stochastic timing can also be performed, in that case,
windows will pop out with the event path, and the status window
will have the combined distribution function. Figure 3 presents
an automaton model in' the environment. The environment also
produces C code for controlling the system under consideration.
In our PN model we have extended the definition of stochastic
timed Petri Nets, to have additional timings. Our model has
three times associated with it, a place time, a delay time, and an
event time (see Figure 4). The place time is a time where the
token is held back and delays the enabling of the transition. This
represents the computation time of that place. The delay time is
a time associated with the input arcs to a transition. It represents
the time to leave the corresponding place. The event time is
analogous to the single time in stochastic timed Petri Nets which
is called firing time. We believe that this lends to a more
intuitive representation of the times and simplifies the modeling

task since it captures more details than the original timed Petri
net model.

of that net. The states are represented as the marking, the events
are just the transitions. The three "times" are pushed into the
events. The system convolves the maximum of the input delays
with the event, and the maximum of the place times. The
maximum b c t i o n is a standard convolution, except that the
maximum is used instead of multiplication.

Place Time

Delay Time

Event Time

Figure 3 A snap shot of the FSM environment

We can define the new model as:
PN = (P,T,A,Kxd
where
P = set of places with associated random variables;
T = set of transitions;
A = A,,, U A,,,/, with
- A,,, set of elements from {P x T} with associated
random variables;
- A,,,,,,set of elements from {T x P} ;
W = a weight function, w:A +{ 1,2,3,... }; and
xII is an initial marking.
The environment for Petri-Nets is similar. Places are
represented graphically by circles, transitions by ellipses, and
arcs by arrows. As mentioned above, there are three locations
where one can place timing information: on the events - the
event time, which is the time the actual event takes; place time when a token is moved through a transition firing there is a place
time which hides the token until it has expired; and the final time
is a delay time which comes into effect when a transition fires it is the time for the event to reach the transition. The event time
will not start until all of its input tokens delay time has expired.
Figure 5 depicts a snap shot of the Petri-Net environment in
action.
The system generates C code for the user hybrid system, so
one can simulate and control an actual system using the code.
The C code is currently generated for FSMs (soon code will be
generated for PNs too). A Petri Net will be converted to a FSM
before code is generated; all of the timing is then placed on the
events. The user has to select the initial state, and provide the
function for simulating/generatingthe events; the code will keep
track of the elapsed simulated time, and will return when it
reaches a state with no transitions.
The environment allows conversion back and forth between
the FSM and PN models. Conversion to a Petri net is straight
forward, but one loses the event probabilities. The only thing
that's needed is to create a transition for every event.
Conversion from a Petri-net to a FSM is only possible if the PN
is k-bounded, which means no place can ever have more than k
tokens. The system generates a state for every possible marking

Figure 5 A snap shot of the Petri-Net environment

The algorithm for generating all of the markings starts with
some initial marking, then goes through all of the possible
transitions. If it can fire, the firing is simulated, and the new
marking is inserted into the set of states. If it is already
represented, the transition is kept; otherwise the transition is kept
and recursion is done with the new marking. This process is
repeated until no transitions can be fired.
Our system can serve as a simple graphical simulator,
analyzer, synthesizer, monitor, and controller for hybrid systems
models using either Petri nets or FSMs high-level frameworks.

4. TRENDS IN MACHINE VISION SYSTEM DESIGN

The predictive model provides a baseline of "in-advance''
information but if routine deviations are greater than can be
tolerated, sensors are needed to augment this baseline
information for feedback to controllers. In existing systems,
estimates of the impact of sensing systems on process
performance indicate as much as a six fold increase in effective
operation speed [7]. A general review of different sensors for
robotics and automation can be found in [8]. One of the major
contributions of information technologies to sensors was the idea
of digitized output, which removed analog variation from the
oumuts. A good illustrative example is machine vision. which
Y

107

Figure 4 The proposed three time zones for a timed Petri net

grows from a standard composite video signal, that the television

industry uses, to a more general-purpose sensor with on-board
intelligence.
However, although it has been well recognized in the past
three decades that vision can add considerably to flexibility by
simplifying grippers, component feeders, and location tooling,
and can reduce the engineering time required to implement it,
the capabilities of commercial vision systems for use in part
verification, kitting, and presentation for robotic assembly are
still very limited. Until the late 1980's, most of the vision
systems employ a camera which outputs a video signal limited
by the traditional TV standard (typically 30 frames per second
specified by the RS170 established in 1950's) and an objectdependent structured illumination. For use as a robot vision
system, a frame grabber board and a high performance host
computer must accompany the video camera. The conventional
vision approach generally discards color information and
requires a substantial amount of memory and data
communication time, and sophisticated vision interpretation.
Variations in surface reflectance, coupled with algorithm
computational demands, often make the conventional approach
too expensive, unreliable, and slow. In addition, the
conventional vision approach, which attempts to emulate human
eyes and brain, does not necessarily yield the accurate data
required by the robots.

4.1 Alternative Vision System Architecture

To overcome the problems associated with the traditional
video-based vision system, several vision systems were designed
for robotic applications. Among these is a Flexible Integrated
Vision System (FIVS) developed at Georgia Tech in the late
1980's [9], which offers performance and cost advantages by
integrating the imaging sensor, control, illumination, direct
digitization, computation, and data communication in a single
unit. By eliminating the host computer and frame grabber, the
camera is no longer restricted by the RS-170 standard and thus
frame rates higher than 30 f p s can be achieved.

digital converter (ADC). The DSP-based control board provides

a direct software control of the CCD array scanning and
integration time, the intensity of the collocated illumination, and
the real-time execution of an user-selectable vision algorithm
imbedded in the EEPROM. In operation, the PLD decodes the
control signals to initiate row shifts and column shifts in
response to commands from the DSP-based control board.
Particular row shifts and column shifts enables retrieving only a
specific relevant area from an image. The PLD also provides
control signals to ADC for performing the analog-to-digital
conversion synchronized with row shifts, and enables the video
buffer when the DSP reads or writes data to the VRAM.

Address and Data Bus

Figure 6 Schematic of a flexible integrated vision system

Architecture
As shown in Figure 6, the central control unit of the flexible
integrated vision system is a microprocessor-based control
board. The design is to have all of the real-time processing
performed using the microprocessor control board without
relying on any other system or computer. The prototype of
FIVS is shown in Figure 7.
On-board processor
The DSP-based control board is designed to communicate with
several option boards in parallel to tailor the system for a
number of applications. Each of these option boards is
controlled independently by a programmable logic device
(PLD), which receives a peripheral select signal, a readwrite
signal, and an address signal from the microprocessor control
board. Typical examples of the option boards for the FIVS are
the digital video head, a real-time video recorddisplay/playback
board, and an expandable memory board.
Camera
The video head consists of a m x n CCD array, the output of
which is conditioned by a high bandwidth amplification
circuitry. The output is then sampled by a "flash" analog-to-

Figure 7 FIVS and its collocated illumination system

Imbedded software
The vision system imbedded software gives users the
flexibility to control the CCD array scanning and integration
time and the intensity of the illumination. With the CCD under
software control, partial frames can be "captured" instead of the
customary fill frame, reducing the cycle time required to capture
and process an image. The ability to shift out partial frames is
ideal for high speed tracking applications where the approximate
location is known from a prior image. By reducing the time to
capture an image, the effective frame rate is increased. For
example, shifting out 1/4 of an image can increase the frame rate
up. to 480 fps, not including the time required for illumination

and image processing. This frame rate is 16 times the rate

achievable from the RS- 170 standard.
4.2 The New Trends

Unlike conventional RS 170-based systems which require

pixel data to be stored in a video buffer before processing of
pixel data can commence, the FIVS design provides an option to
completely by-pass the video buffer and thus offers a means to
process andor to store the digitized pixel data by directly
transferring the ADC output to the DSP. For real-time visionbased object tracking and motion control system applications,
the scheme represents a significant saving in time and video
buffer size required for processing an image. As an illustration,
consider an image array of m x n pixels. The time needed to
store the entire image (with no computation) in a memory at K
MHz is (m x n)lK ps and requires (mxn) bytes of memory.
Typical array size of a CCD ranges from 200x160 to 4096x4096
of pixels. The corresponding video buffer and time required
simply to store the entire image at a clock rate of 10 MHz would
range from 32K btyes to 16M bytes and 3.2 ms to 1600 ms
respectively! Clearly, the option to completely by-pass the
video buffer offers a potentially useful solution to eliminate the
frame storage prerequisite which is often required in
conventional vision systems. Furthennore, this scheme
completely eliminates the special hardware needed in acquiring
the digitized pixel data for storage.
Auulications
With the on-board intelligence, computer controlled
machine vision systems have found a number of real-time
applications, where the accuracy of image gray-scale pixel
values far outweighs image. Some of these examples are robotic
part pickup [lo], motion tracking [ 111, three degrees-of-freedom
orientation sensing [ 121, servo-track-writing in hard disk drive
manufacturing [ 131, disassembly automation [ 141, and haptic
sensor [ 151.
The advance in direct-digital machine vision will continue
to lead to new ways of addressing industrial automation
problems that were difficult (if not impossible to solve),
particularly for the traditional industries. One such example is
an on-going development of a high-speed live-bird handling
system for poultry processing applications, where machine
vision has played a significant role in automating the process of
transferring live birds from a moving conveyor to shackles,
typically at a line speed of 3 birds/second. Live-bird handling
problems have been found difficult because the birds tend to flail
about when they are caught. Non-evasive techniques must be
developed along with the study of stimulus environments to
promote behavior compliance, the study of the role of visual
responsiveness, and the evaluation of vision acuity in different
spectral environments. Often such real-time control application
requires a stringent combination of structured illumination,
reflectance, and imaging sensor.
Figure 8(a) illustrates how structured illumination and
reflectance of a machine vision system could play a significant
role in the live-bird handling application [ 161, where the bird's
posture is determined for real-time manipulation of the bird's
legs. In order to keep the bird from flailing and its presentation
uniform as the bird enters the grasper, retro-reflective sensing
technique [IO] has been used to obtain a snap shot of the moving

bird. The structured illumination system consists of a lowintensity spectrally filtered illumination that insensitive to the
bird, and a vision system that captures a snap shot of the bird
against a retro-reflective background. Figure 8(b) shows an
image of a bird on a conveyor moving at 0 . 5 d s toward the
grasper. The image of he bird was captured against a 580-85
Black Scotchlite retro-reflective background with a low-intensity
illumination filtered with a Roscolux full-blue filter since birds'
are insensitive to blue light (or low 400nm wavelengths).

t=O

Beam-switch
detection of incoming
bird,computingbird i
size and posture from

t=t,

t=t,

Fingers begins
grasping and
translating the
bird

Fingers relieve
the bird and
conveyor feeds
Be bird F: feet
into shackle

imaging

(a) Illustrating schematics

Figure 8: An example with structured illuminatiodreflectance

189

Advances in New Imaging, Sensors

In the early 1990's, CMOS sensors emerged as low-cost,
low-power alternatives to CCD. The principle architectural
solutions, which enabled the high data throughput, are the
effective integration of pixel readout processing, ADC
conversion, and the high-speed dual-port RAM in one single
chip. The core of the CMOS sensor designs is a m x n
photodiode active pixel array, which is accessed in row-wise
fashion and readout into a column ADC's in parallel. With the
addition of a column of dual-port SRAM's, the readout of the
digital data can be done during the A/D conversion of the next
row. The sensor has an on-chip digital block, which runs the
row processing, ADC conversion, and readout and allows

flexibility in selecting rows and columns as well as defining the

start time for row processing or read. The use of parallel pixel
readout and digitizing, as well as easy ways of multiplexinddemultiplexing data - techniques required for high-speed large
format sensors - is challenging CCD technology in mainstream
applications. Today, CMOS sensors, for example the 1024x1024
CMOS Active Pixel Sensor [17], has the potential to achieve
very high output data rate over SOOMB/second and a low power
dissipation of 350mW at a clock rate of 66MHz.
Attempts to emulate human visual perception have led to the
development of high mamic-yange color (HDRC) imaging
systems [ 181. The power of human visual perception lies in its
very high dynamic range, its robust object detection due to high
and constant contrast resolution in both bright and dark regions
of a scene. Natural photoreceptors like those in our eye have a
logarithmic response that detects very fine absolute steps in the
dark or shade while they limit their response to larger absolute
steps at high intensities. HDRC CMOS pixel generates an
output voltage equivalent to the log of the local optical intensity.
As a result, the high dynamic-range of the HDRC camera
outperfom the digital CCD camera that has reached its limits in
spite of its advantage in resolution and all its post-processing
power.
ComDutinP and Intemtion
The recent introduction of microprocessors with large
internal caches and high-performance external memory
interfaces make it practical to design high-performance imaging
systems with balanced computational and memory bandwidths.
Sgro et a1 [ 191 presents a fiamework that allows a developer to
choose a microprocessor system that offers the perfonnance and
scalability often required by a real-time vision application.
Using the component inspection application as an example, they
demonstrate that coprocessor-based solutions with local memory
architects allow throughput to scale linearly as the number of
processors increase.
For demanding vision applications
especially those require fimre expansion, the most practical
solution remains a co-processor board that is more scalable, has
higher throughput, and ultimately is cheaper that the native
solution.
Finally, one other potential impact is the influx of low-cost
USB and Firewire cameras into the lucrative consumer market
that drives the development of the USB E201 and IEEE 1394 [21]
(commonly known as Firewire) communication standards. The
USB standard was designed to replace typical parallel and serial
I/O ports (such as RS232) and has widely accepted by the PC
industry. Future USB 2.0 is expected to have a speed over 120240 Mbps. The IEEE 1394 standard was designed as a highspeed bus with digital video as its target application. The bus
currently runs at speeds up to 400 Mbps and expects to exceed
1600 Mbps in the near future.
5. PROTOTYPING MACHINE VISION DESIGN
Imaging sensors are characterized by their specific
bandwidths or wavelengths of light, which maximize the sensor
response and will provide it an optimum operating environment.
It is desired that the photo-detector respond only to the light
from the illumination source structured for the object but not that
of ambient lighting. Synthetic images [22] can efficiently be

used to study the effects of illumination and vision system

design parameters on image accuracy, providing insight into the
accuracy and efficiency of image-processing algorithms in
determining part location and orientation for specific
applications, as well as reducing the number of hardware
prototype configurations to be built and evaluated. Figure 9
compares the processes used to generate synthetic images for (a)
photo-realistic and (b) physically-accurate synthetic images for
vision system applications.
As shown in Figure 9, an accurate mathematical model is
needed to describe the physical scene and the vision system used
to capture that scene. This model is used to simulate scene
illumination, which is represented as an array of [pixel]
radiances. This array of radiances is then converted to
energy/area values, which are transformed by a mapping based
on a model of the system sensor and how it converts incident
light energy into gray-scale values.

Mapping based on
Vision Sensor

Simulated Vision

(a) Photo-realistic

(b) Physically-accurate

Figure 9 Model of the synthetic imaging process

The physically-accurate synthetic image is simulated in a
two step process. In the first step, RADIANCE, a freelydistributed software package from the Lighting Systems
Research Group of the Lawrence Berkeley Laboratory, is used to
solve the radiative heat transfer equation. In the second step, the
sensor model for the computer vision system is modeled using
the power law [lo].
Figure 10 quantitatively compare various methods of
generating synthetic images, where synthetic images of the
retroreflective background were generated and compared to a
captured image of the retroreflective field (Figure IO). As seen
in Figure lO(a), a CAD-generated image assuming an ideal
diffise surface results in an image which is nearly black. Figure
1O(b) illustrates RADIANCE 's ability to model the retroreflective
background; however, the illuminated area is too small and too
sharply defined. Incorporation of the finite aperture (Figure
10(d)) results in an image with a more acceptable transition
between the illuminated and non-illuminated areas, but the

190

illuminated area is still too small. The importance of accurate

source emission distribution modeling is shown in Figure lO(c).
Other detailed illustrative examples of using physically-accurate
synthetic images can be found in [23].
The benefits of this realistic image synthesis are three-fold.
First. it provides a rational basis for designing the hardware and
software components of a machine vision system. Secondly, it
provides a standard platform for comparing algorithms and
predicting the optimal algorithm (and optimal performance) for a
specific application. Additionally, it provides an opportunity to
perform an in-depth study of the factors that can significantly
degrade the performance of image-processing algorithms and aid
in the determination of critical design parameters. A third
benefit is the ultimate development of a well-designed CAD-tool
which utilizes physically-accurate synthetic images to accurately
and inexpensively predict the performance of a proposed vision
system design prior to implementation or the construction of a
prototype, minimizing the need to build and test a large number
of hardware configurations. Such a tool also allows necessary
changes in part design to be made earlier in the design phase,
significantly reducing implementation time and improving
industrial reliability

Figure IO. Synthetic images of Retroreflectivefield -Clockwise from

upper-left-hand corner: a) generated with a CAD-package reflectance
model, b) RADIANCE-generated image, using infinitesimal pinhole and
cone-angle illumination model, c) image with finite pinhole model and
Gaussian illumination distribution, d) finite pinhole, cone-angle
illuminationmodel.
5. CONCLUSIONS
We have presented trends of prototyping design and
automation with an emphasis on the following two subtopics.
The first is a brief review of Discrete Event and Hybrid Systems
prototyping. A simple software environment system was
developed for simulating, analyzing, synthesizing, monitoring,
and controlling discrete event and hybrid systems.
The second is a review and the trends on prototyping realtime machine vision system design. Specifically, we present an
alternative system design to overcome those problems associated
with a traditional video-based vision system, which is a
physically-accurate image synthesis method as a flexible,
practical tool for examining a large number of
hardwarelsoftware configuration combinations for a wide range
of parts.

REFERENCES
[I] Balemi, S. Control of Discrete Event Systems: Theory and
Application. Ph.D. thesis, Automatic Control Laboratory, Swiss
Federal Inst. of Tech. (ETH), Zurich, Switzerland, May 1992.
[2] Ho, Y. Discrete Event Dynamical System and its Application to
manufacturing. In IFAC Cong. Proc. (1 98 1).
[3] Y.C. Ho and X.-R. Cao, Perturbation Analysis of Discrete Event
Dynamic Systems. Boston: Kluwer Academic Publishers, 199I .
[4] G.J. Olsder, "Applications of the theory of discrete event systems
to array processors and scheduling in public transportation," in
Proc. of 28th IEEE Conf. on Decision and Control, Dec. 1989.
[5] Sobh, T., Bajcsy, R., and James, J. Visual observation for hybrid
intelligent control implementation. In 3 1st IEEE CDC (Tucson,
AZ,USA, December 1992), IEEE.
A discrete event
[6] Sobh, T., Jaynes, C., and Henderson, T.
framework for intelligent inspection. In Proc. 1993 IEEE Int.
Conf. Robotics and Automation (May 1993). Atlanta, GA.
[7] Eversheim, W., 1991. "Strategies and Tools meeting Future
Challenges in Manufacturing," Int. Conf. on Appl. of
Manufacturiing Technology, SME, Apr. 17-19, Washington D. C.
[8]Lee, K.-M., "Sensors And Actuators," Section in Chapter on
Robotics, CRC Handbook of Mechanical Engineering, CRC
Press, 1998.
[9] Lee, K.-M. and Blenis, R., "Design Concept And Prototype
Development Of A Flexible Integrated Vision System," Journal
Of Robotic Systems, 11(5),387-398, (1994).
[IO] Lee, K.-M., "Design Concept Of An Integrated Vision System
For Cost-Effective Flexible Part-Feeding Applications," ASME
Trans. Journal Of Engineering For Industry, Vol. 116, pp. 421428, November 1994.
111 Lee, K.-M. and Qian, Y.-F., "Development Of A Flexible
Intelligent Control For Target Pursuit," ASME Trans. Journal of
Manufacturing Science and Engineering, Vol. 120, August 1998,
pp. 640-647.
121 Lee, K.-M., Meyouhas, G., and Blenis, R.,"A Machine-VisionBased Wrist Sensor For Direct Measurement Of Three DegreesOf-Freedom Orientation," Mechatronics, Vol. 3, No. 5, pp. 57 1587, 1993.
131 Gamer,H. K-M. Lee, L. Guo, "Design Analysis of a Grating
Interferometer Sensor for HDD Servo-Track Writing,"
IEEE/ASME AIM'99 Proceedings, Atlanta, GA Sept. 18-23, 1999.
[I41 Presti, Peter, "The Haptic Lens - a Tactile Sensor," IEEElASME
AIM'99 Proceedings, Atlanta, GA September 18-23, 1999.
[ 151 Lee, K.-M. and Bailey-Vankuren, M., "Supervisory Control of an
Automated Disassembly Workcell Based Blocking Topology,"
Proceedings of the 1997 IEEE International Conference on
Robotics and Automation, Albuquerque, New Mexico.April 2025, 1997.
[ 161 K.-M. Lee, "Design Criteria for Developing an Automated LiveBird Transfer System," IEEE International Conference on
Robotics and Automation, San Francisco, 22-28 April 2000.
[ 171Zhou, Z., B. Pain, E. R. Fossum, "CMOS Active Pixel Sensor with
On-Chip Successive Approximation Analog-to-Digital Converter,"
IEEE Trans. on Electron Devices, pp. 1759-1763, Oct. 1997.
[IS] Seger, U., H.-G. Graf, Landgraf, "Vision Assistance in Scenes with
Extreme Contrast," IEEE Micro 13(I ) , February 1993, S. 50-56.
[ 191 Sgro, J., P. Stanton, and J. Delfino, "Evaluation Framework for
Scalable Microprocessor Vision Systems in an industrial
Inspection Application," Alrcon, Inc.
[20] "USB Press," http:!hww.usb.oreimess, June 1999.
[2 I] Intel, 1 394 Technology," Ii~~i~:~iiiz~elo~c~.in~el.conl~~echnolouv:
I 79.1,
June 1999.
[22]Parker, J. and K.-M. Lee, "Physically-Accurate Synthetic Images
for Machine Vision Design," ASME Trans. Journal of.
Manufacturing Science and Engineering, November, 1999.

191

"