ARTIFICIAL INTELLIGENCE IN IMAGE PROCESSING

P. Gilmore
John F.
Artificial Intelligence Branch
Artificial Intelligence Branch
Georgia
Georgia Tech
Tech Research
Research Institute
Institute
Atlanta,
Atlanta, Georgia 30332

Abstract
Image processing technology concentrates on the the development of data extraction
techniques applied toward the statistical classification of visual imagery. imagery. in classical
In classical
image processing systems,
systems , an image
image is
is [1]
[1] preprocessed
preprocessed toto remove
remove noise,
noise, [2]
[2] segmented to
produce close object boundaries, [3] [3] analyzed
analyzed to
to extract
extract aa representative feature vector,
vector,
and [4]
[4] compared
compared to
to ideal
ideal object feature vectors by by aa classifier
classifier to
to determine
determine the
the nearest
object classification and its its associated confidence level. This type of of processing
attempts to formulate aa two-dimensional
two -dimensional interpretation
interpretation of three-dimensional
of three -dimensional scenes using
scenes using
local statistical analysis, an entirely numerical process. process. Symbolic information
information dealing
relationships, object attributes, and physical constraints is
with contextual relationships, is ignored
ignored inin
such an approach. This paper describes a number of artificial intelligence techniques
which allow symbolic information to to be exploited inin conjunction
conjunction with
with numerical
numerical data to
improve object classification performance.
improve performance.

Introduction
Image processing isis aa technology
technology long
long entrenched
entrenched in
in the
the development
development of statistical
vision systems. Applications in
in the
the areas
areas of
of target
target recognition,
recognition, bio-
bio-medical
medical analysis,
scene matching, and manufacturing have spurred
spurred the development of a number of of image
processing systems specifically tailored
tailored to
to address
address individual
individual problem characteristics.
The classical
classical approach to image processing (Figure
(Figure 1)
1) adhered to by most of these systems

-
consists
consists of
of four
four processing stages:
stages: preprocessing, segmentation, feature extraction,
extraction, and
classification.

Preprocessing
Preprocessing "'! Segmentation
Segmentation Feature
Feature ** Classification
Classification joi.
Extraction
Extraction

o Noise
o Noise removal
removal o Edge
o Edge detection
detection oo Statistics
Statistics o Class
o Class separation
separation
oo Edge linking
Edge linking oo Shape
Shape metrics
metrics o Distance
o Classifiers
Distance Classifiers

1. The Classical Approach To Image Processing

Figure 1.
A raw
raw image
image is
is preprocessed
preprocessed upon
upon entering
entering the
the system
system to
to remove
remove the
the effects
effects of
of noise
due
due to
to sensor
sensor variations. An edge enhancement
enhancement operator is then convolved with the image
operator is image in
in
order
order to
to strengthen
strengthen object
object boundary information.
information. Segmentation manipulates aa series
series of
of edge
edge
detection operators to produce near complete
complete boundary
boundary lines.
lines. These boundary lines are
closed through thethe use
use of
of edge linking or degapping algorithms. Extraction algorithms
which compute
compute statistical
statistical and
and structural
structural features
features are
are run
run in
in order
order to
to construct
construct aa feature
vector each segmented
vector representation for each segmented object.
object. Classification compares
compares the
the image
image extract
extract
feature vectors to to ideal
ideal object
object vectors using distance classifiers
classifiers to determine class
separation. Image
Image objects
objects are
are classified based on their closeness
closeness to
to ideal
ideal object
object vectors
vectors
and aa normalized classification confidence
and confidence value is computed to
value is to indicated
indicated the level
level ofof
certainty.

Though widely used through the years, several shortcomings exist in in aa purely
statistical classification. First, most segmentation
statistical approach to object classification. segmentation algorithms
algorithms assume
assume
that
that the objects being segmented will be
be of
of aa uniform
uniform intensity
intensity nature.
nature. This assumption
assumption isis
in light of
invalid in of the
the bi
bi-modal
-modal characteristics
characteristics associated with the
the majority
majority of
of real
real world
world
object,
object, such as cars in which engines may possess radically
radically different
different intensities
intensities compared
compared
192
192 //SPIE
SPIE Vol.
Vol. 528
528 Digital
Digital Image
Image Processing (1985)
(1985)

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/11/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

 to the rest of the vehicle. Second, statistical
to statistical comparisons
comparisons are void of
of the existence of
of
sensor-induced
sensor- induced image noise. As feature extraction algorithms are not capable of
eliminating noise affects, an object's feature vector
eliminating vector statistics may only be sixty to to
eighty percent accurate. This error
eighty error margin propagates into an unacceptable range
range when
when aa
large number
number of
of features are used (most
(most system are on the order of 60 or more features).
Third, the arbitrary assignment of weights to each feature being evaluated is is data
dependent and not robust enough to handle additional data
not robust data sources. Fourth, algorithm
parameters are largely database dependent and hardwired even though the parameters
themselves may be of
themselves of an
an adjustable
adjustable nature.
nature. Though the
the problem of
of feature set selection
selection is
is
difficult, no
difficult, no one fully understands how each algorithm parameter should be adjusted to to
improve
improve system performance.

The goal of this paper is

is to present a number of artificial intelligence techniques
that can be used to overcome current image
image processing shortcomings and improve vision
system performance. Several approaches along these lines are discussed in
Several in the
the following
sections including image understanding methods, knowledge base and expert systems,
knowledge representations,
knowledge representations, inexact
inexact reasoning,
reasoning, and
and heuristics
heuristics in low level processing.
processing.

Image
Image Understanding
Image understanding combines the data extraction techniques of imageimage processing
processing with
the information exploitation techniques of artificial intelligence in in order to fully
interpret aa three dimensional scene (Figure 2).
2) . Object recognition in in an
an image
understanding system integrates four types of processing: two-dimensional
two -dimensional analysis,
temporal processing,
temporal three-dimensional
processing, three -dimensional processing,
processing, and
and knowledge
knowledge base
base processing.
processing.
Two-dimensional
Two- dimensional analysis complements the statistical classification of of objects
previously described with information on local object interrelationships.
interrelationships. After anan object
object
has been segmented and classified, its its local
local context
context is
is analyzed
analyzed to
to provide
provide positive
positive or
negative support to to its
its initial classification. For example, consider the primary
statistical classification of an object as aa vehicle. Examining it's local
interrelationship oror context may provide information
information that the vehicle isis located
located within
within aa

2D ANALYSIS

oo feature
feature

oo interrelationships
interrelationships

4
IMAGE'
IMAGE PREPROCESSING
PREPROCESSING SCENE OBJECT
HISTORY
HISTORY RECOGNITION
RECOGNITION

oo motion
motion

3D
3D ANALYSIS
ANALYSIS KNOWLEDGE
BASE
BASE
o stereo
o stereo vision
vision
o surface
o surface orientation
orientation
oo shape
shape
oo interrelationships
interrelationships

Figure 2.
2. An Image
Image Understanding
Understanding System Prototype

SPIE
SPIE Vol.
Vol.528
528 Digital
Digital Image
ImageProcessing
Processing(1985)
(1985)// 193

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/11/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

 lake. invalid context
This would be an invalid in support
context in the primary
of the
support of classification, but may
primary classification,
be positive support ofof aa secondary classification of of rock. Interchanging the
conflict free
classification labels would then produce a conflict free representation the object
of the
representation of in
object in
question. Repeating this all objects
for all
this process for objects in improves upon
in aa scene improves the previously
upon the
computed statistical and provides
classifications and
statistical classifications more valid
provides aa more representation scheme
valid representation to be
scheme to be
used in
used stages.
in the three other processing stages.
Temporal processing
Temporal and exploiting
maintaining and
processing through maintaining history allows
scene history
exploiting scene the system
allows the system
guided by
be guided
be by information it learned from
it has learned images.
previous images.
from previous For example, if an an object
as aa tree
sensor as
was classified by aa stationary sensor in image
tree in with aa confidence of
-1 with
image-1 99%,
of 99%,
looking for a a tree in same location
in the same in image
location in -2 would be an appropriate action.
image-2
the system to
allows the
Maintaining aa scene history also allows to analyze sequence of
analyze aa sequence of images for
motion attributes [1]. Motion detection
[1]. determines the
detection determines location of
the location of moving object in an
moving object
image, their speed and velocity, andand an information is
overall motion confidence. This information
an overall is
fused into
fused the current object classification where it
into the functions. First, it
three functions.
it performs three
in the
aids in
aids (e.g., aa tree
classifications (e.g.,
the validation of the initial object classifications moving at
tree moving at 30 mph
30 mph
tree). Second, it
is not a tree).
is confidences, thereby having
it improves vehicle classification confidences, having a
object propagation
dramatic effect on validated prioritized object (i.e., the delta of moving
propagation (i.e.,
vehicle is global scene analysis by the
is quite measurable). Third, the global the context analyzer
impact on individual
recognizes motion as having a major impact target classification as
individual target as well
well as
potential target array analysis. fact that an object has aa consistent
The fact velocity and
consistent velocity
direction over invalid classifications
over time eliminates invalid such as
classifications such rock or
tree, rock
as tree, road.
or road.
analysis interprets
Three -dimensional analysis
Three-dimensional sequence of
interprets aa sequence (usually from
images (usually
of images variety of
from aa variety
angles) in
sensor angles)
sensor order to
in order depth, orientation and range
to determine depth, information. Algorithms
range information.
for this
used for stereo vision [2],
this type of processing address stereo [3], and
orientation [3],
[2], surface orientation
shape -from shading
shape-from [4]. Global
shading [4]. context of
Global context an object
of an is also
object is extracted and analyzed in
also extracted in this
this
phase. Whereas local provides the
has provides
local context has the fact
with the
the system with fact that in the
rock is in
that the rock the
lake, global context indicates
indicates that the lake is is in field, next to aa mountain, and
in a field,
surrounded by trees.
Knowledge base processing
Knowledge base all of
analyzes all
processing analyzes the above
of the information to
above information produce an
to produce an accurate
accurate
understanding of
understanding representation, typically through
of aa scene representation, rule-based
production rule
through production analysis.
-based analysis.
programs in
systems differ from conventional programs
Production systems in two respects. The first is
major respects.
two major
that the
that systems use
the production systems method for
different method
use aa different the state
encoding the
for encoding computation.
of aa computation.
state of
conventional program encodes the
A conventional the state assigning values
by assigning
state by local and
to local
values to variables.
global variables.
and global
A production encodes the
production system encodes by putting
state by
the state expressions in
putting expressions in the global working
system's global
the system's working
memory. The other difference between production systems and conventional programs is is the
way flow of control is is managed. A conventional program uses sequential execution of
program uses
including subroutine calls, loops, and
constructs including
statements plus a number of control constructs
conditional satisfaction. Each production's
conditional branching. A production system uses antecedent satisfaction. production's
is aa description of the states in
antecedent is
antecedent in which the production rule is applicable; the
rule is
antecedent becomes true when there is is some information in working memory that the
can process.
production can the interpreter
process. When the the match
performs the
interpreter performs process, it
match process, it is, effect,
in effect,
is, in
for a production that knows how to process the
searching for
searching that is
data that
the data the symbolic
in the
is in scene
symbolic scene
representation working memory. When it it finds that production
finds that executes, its
and executes,
production and its consequent
symbolic representation is cycle, the interpreter performs the
changed. On the next cycle,
is changed. the match
again to
again find a production that can handle the new data.
to find
A production rule system in
rule system purest form
its purest
in its series of
is aa series
form is antecedent-consequent
of antecedent rules
-consequent rules
global database for
which search a global preconditions and
prioritized preconditions
for prioritized and are sequentially chained
are sequentially
by consequent actions towards
by goal. The
decision goal.
final decision
towards aa final knowledge-based
majority ofof knowledge
The majority vision
-based vision
systems
systems in existence today implement some type
in existence type of aa production rule strategy.
rule strategy.

Knowledge Base Systems

knowledge base
The design and development of aa knowledge
The system is
base system major software
is aa major undertaking
software undertaking
to the
due to
due required to
the breadth and depth of knowledge required prototype, as well as
develop aa prototype,
to develop as the
extensive knowledge validation during knowledge
extensive acquisition required
knowledge acquisition for system
required for refinement.
system refinement.
A knowledge system is aa level of predetermined
level of in aa specific problem
confidence in
predetermined confidence
mechanics of
domain. The mechanics transparent to
are transparent
of such aa system are user, causing
the user,
to the knowledge base
causing knowledge base
occasionally be
system development to occasionally viewed by
be viewed non-implementors
by non as aa simple task.
-implementors as In
reality, the
reality, multi-faceted
the multi- of aa knowledge
aspect of
faceted aspect it one
base system actually makes it
knowledge base the
of the
one of
types of
most difficult types systems to
computer systems
of computer construct.
to construct.
systems offer
base systems
Knowledge base deal of
great deal
offer aa great in assisting
utility in
of utility in aa variety
humans in
assisting humans
of
of knowledge base
domains. First, the knowledge base system building process itself develops human
knowledge (normally
expertise. The explicit codification of knowledge implicit and
(normally implicit is an
elusive) is
and elusive)
illuminating process
illuminating insights within
process which leads to many new insights domain. In some
particular domain.
within aa particular
cases, the knowledge gained in in order to build a knowledge base system makes existing
order to

194 //SPIE
194 SPIE Vol.
Vol. 528
528 Digital Image Processing (1985)
Digital Image (1985)

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/11/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

 knowledge inexpensive
knowledge and available.
inexpensive and today's society
In today's
available. In , the
society, hard problems
the hard require the
that require
problems that the
best (and
best expensive) human expertise are greatly proliferating as
most expensive)
(and most technology becomes
as technology
more complex.
more places extensive
complex. A knowledge base system places solving knowledge
problem solving
extensive problem knowledge inin the
hands less-trained,
of less-
hands of trained, lower -salaried workers. Third,
lower-salaried knowledge base systems in large
problem domains are capable of performance in excess of
in excess humans. They
of humans. capable of
are capable
They are dealing
of dealing
are known
that are
with large solution spaces that be hard
to be
known to humans, and thus extend
for humans,
hard for type
the type
extend the
can be
and degree of problems that can solved.
be solved.
The actual construction of of aa knowledge base system requires three distinct
specialist in
individuals. First and foremost, aa specialist the specific
in the domain (i.e.,
application domain
specific application
computer vision)
computer vision) must be designated. is an
specialist is
designated. A specialist individual who
an individual is knowledgeable
who is knowledgeable in in
area but
a particular area
a the experience
lacks the
but lacks to build
required to
experience required expertise. His role is to
build expertise. to
(such as image
identify applicable domain knowledge (such
identify algorithm suites)
image processing algorithm and aid
suites) and in
aid in
process to
decision process
the development of the decision embedded in
be embedded
to be the system.
in the programmer
system. Second, aa programmer
is tasked
is the coding of the domain knowledge is
with the
tasked with is required. communicates
required. The specialist communicates
through the
with the programmer through third team
the third the knowledge engineer. The knowledge
member, the
team member,
the key role in
plays the
engineer plays cycle. He must be
system development cycle.
in the knowledge base system
knowledgeable in the domain being modelled, capable
knowledgeable in of interacting
capable of and extracting
interacting and heuristic
extracting heuristic
knowledge from the specialist, in artificial intelligence techniques of
proficient in of
knowledge representation and exploitation, and capable of presenting all of this
information atat a level
level understandable by the programmer so so that is properly machine
it is
that it
implemented. In the majority of vision systems
In being developed today, the knowledge
systems being
and domain specialist are one in
engineer and
engineer same.
the same.
in the
can be
construction can
Knowledge base system construction broken down
be broken into three
down into phases:
three phases:
1)
1) analyze domain knowledge in order
knowledge in devise aa detailed
to devise
order to formal knowledge
detailed formal knowledge
for the
representation scheme for construction of
the construction the knowledge
of the base,
knowledge base,
2)
2) dependent control
implement an application dependent strategy as
control strategy computer sys-
as aa computer sys-
behavior in both tested and
intelligence behavior
tem capable of exhibiting intelligence un-
and un-
tested situations, and
tested situations, and
3)
3) refine the system
refine the in order
system in to make
order to it consistent
make it proficient within
and proficient
consistent and within
iterate, iterate,
(i.e., iterate,
a specific (i.e., iterate)
iterate, iterate)
components a global
A knowledge base system consists of three components base, aa knowledge
global data base,
base, and aa control strategy. TheThe global data base or symbolic scene representation
contains image information that has been extracted from front end image processing
consists of
algorithms. This information consists local entity
of local attributes, external
entity attributes, data, and
external image data, and
global entity relationships.
global The global data base also as a working memory when
functions as
also functions when
image has been fully analyzed it
imagery. When an image
the system is processing imagery. is transferred
it is
it provides
to the knowledge base where it information on
provides information history.
scene history.
on scene
The knowledge base contains all of
contains all domain specific
of domain information required for image
specific information
analysis. entity that
Each possible entity appear in
may appear
that may image (e.g.,
an image
in an vehicle ,,
(e.g., tree, field, vehicle
bridge, knowledge base
etc) maintained by the knowledge base in prioritized fashion. The individual
in a prioritized
represented in a hierarchical knowledge
attributes for each entity are defined and represented
possess attributes identifying
representation. For example, aa vehicle would possess
representation. type,
its type,
identifying its
context. Three types of attributes are maintained:
direction, velocity and context.
location, direction, local
maintained: local
individual object
attributes used to define individual characteristics, global
object characteristics, global attributes concerned
with defining its overall
defining its and decision attributes
overall image posture and attributes inin the form of heuristics
by an
used by
used opinion as
an opinion
an expert in forming an normalcy or
scene normalcy
to scene
as to deviation.
or deviation.
strategy is
The function of the control strategy is to the knowledge contained in
apply the
to apply the
in the
information collected in
knowledge base to the information in the symbolic scene representation order
in order
representation in
to formulate an accurate analysis of the current image. image. Once this process has been
identified in
for each object identified
completed for the image,
in the used to
is used
image, aa decision strategy is further
to further
enhance The decision
the current representation. The
enhance the typically an
is typically
strategy is
decision strategy inference engine
an inference engine
expert-derived
comprised of expert- heuristics to
derived heuristics image. Separately maintaining the
analyze the image.
to analyze the
knowledge, information, and control aspects allows allows maximum system flexibility while
allowing the
allowing expanded. This not only adheres to the true
easily expanded.
the system to be easily concept of an
true concept
expert system but also permits the parallel development of each component once initial
and attributes have been
entities and
entities defined.
been defined.
solving can
employed by knowledge base systems in problem solving
In summary, the methods employed be
areas:
into three distinct areas:
divide into
[1]
11] analysis of global database information,
database information,
[(2]
[,2] the knowledge base,
extensive solution space representation in the base, and
[3]
[3] high level reasoning through the execution of strategy.
of the control strategy. The

SPIE Vol.
SPIE Vol.528
528Digital
Digital Image
ImageProcessing
Processing(1985)
(1985)/ / 195

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/11/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

 USER
5EF

Natural
Language
Interface
t
Control
Structure*
Structure
o control strategy
o rule interpreter

Knowledge Base
'Knowledge Base 1 Global Sensory
Data Base Information
source
o knowledge source
o knowledgerules o symbolic o Input data
rules
o inference rules information
representation

3. Knowledge Base System Architecture

Figure 3.
The development of a knowledge base system
system differs from that
that of an
an expert
expert system
system in
several ways. First, a recognized
recognized expert who has acquired numerous heuristics
heuristics concerning
the most appropriate
appropriate ways to function in a given problem domain is required
ways to required to
to replace
replace the
domain specialist (i.e., specialist is is not the equivalent of an expert). Second, the
reasoning
reasoning mechanism
mechanism implemented
implemented by
by the
the control
control strategy
strategy is
is enhanced
enhanced with heuristics
heuristics so
so that
that
it closely
it closely models expert. Third, the role of the knowledge engineer is
models the expert. is increased
increased to
incorporate
incorporate knowledge extraction and
and acquisition.
acquisition. In In simple
simple terms,
terms, a knowledge base system
is
is an
an expert
expert system
system without
without the
the expertise
expertise provided
provided by
by the
the domain
domain expert.
expert. Figure 3. outlines
Figure 3. outlines
architecture of
the high level software architecture of aa knowledge
knowledge base
base system.
system.

Inexact
Inexact Reasoning

Advanced vision systems engage inin two sorts of high level activities: goal goal achievement
achievement
and belief maintenance. Goal achievement is is an active, knowledge-driven,
knowledge- driven, foreground
activity that
activity that directs
directs system processing
processing toward the
the achievement
achievement ofof predetermined goals.
goals. For
example, the
the control
control structure of aa model-
model-driven
driven vision
vision systems
systems selects
selects rule
rule firings
firings based
based
on how closely aligned the final deductions are in relation to the model being
investigated. Belief maintainence
maintainence [5]
[5] is
is a passive, data-driven,
data -driven, background that
background activity that
keeps beliefs consistent in in light
light of
of system analysis. Suggestive evidence, certainty
factors
factors and
and inexact fall into
inexact reasoning methods fall into this
this category.
category. As an example,
example, consider the
inexact
inexact reasoning
reasoning concept employed in
in the
the TESS
TESS [6]
[6] system.
system.
recursive confidence
TESS uses a recursive confidence evaluation
evaluation scheme
scheme loosely
loosely based
based on
on the
the Dempster-
Dempster-Shafer
Shafer
approach, except that
that the initial
initial seed
seed confidences
confidences are
are provided a statisical
statisical classifier.
Contextual information exists inin one
one of
of three
three forms:
forms: positive evidence
evidence which increases
increases an
object's classification confidence, negative evidence which decreases an object's
classification confidence, or zero evidence which has no effect on an object's
confidence. In
classification confidence. In TESS
TESS positive
positive evidence isis denoted byby 1,
1, negative by -1,-1, and
zero by O.
0.
Suppose that
Suppose that an
an object
object has
has been
been detect
detect in
in an
an image
image and its context has
and its has been identified
as being on a road, in
in a column, and in in motion. The object's initial statistical
is given
classification confidence is given as:
as:
(Object-13
(Object-13 is a (Vehicle .63)
. 63)
(Tank .57)
. 57)
(Clutter .30)
.30)
(Unknown .16)
.16)

The following production rule

rule would
would be
be triggered:
triggered:
(Object
(Object Rule 198
198 (If
(If (Object is on aa road)
(Object road)
(Object
(Object is
is in motion)
(Object
(Object is
is in a column)
(Then (Object
(Object is
is a target)
(RES tank medium)))
(RES medium)))

196 //SP
SPIE Vol. 528
/E Vol. 528 Digital
Digital Image Processing (1985)
(1985)

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/11/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

 The recursive
The recursive evidence scheme is informed that the primary object
scheme is classification
should
should bebe that
that of
of a tank with
with aa medium confidence level. Object-13's
Object -13's classification
confidences
confidences are
are then
then computed as follows:
follows:
RES == [object
[object confidence
confidence (OC)
(OC) ++ (OC
(OC* * (1(1-OC)
-OC) ** evidence ** strength)]
strength)]

(.63 ++ (.63
RES vehicle = (.63 (.63 ** (1
(1-.63) 0 ** .67))
-.63) * 0 .63
.67)) = .63
RES tank = (.57 ++ (.57
(.57 ** (1
(1-.57) * 11 ** .67))
-.57) * = .73
.67)) _ .73
RES clutter = (.30 ++ (.30
(.30 ** (1-.30) *-l
(1 -.30) * * .67))
-1 * .16
.67)) = .16
RES undefined= (.16 + (.16
(.16 ** (1
(1-.16)
-.16) *-l
* -1 * .07
* .67))= .07

where negative values are set to

to zero.
zero.
The
The new classification confidences
confidences are
are thus:
thus:
(Object-13 is a (Tank
(Tank .73)
.73)
(Vehicle .63)
.63)
(Clutter .16)
.16)
(Unknown .07)
.07)

This process is
is repeated
repeated for
for each
each applicable
applicable rule
rule firing
firing that
that would
would occur in the
system.

Cueing
Contextual Target Cueing
Two expert
Two expert vision
vision systems
systems which attempt
attempt to
to address
address the
the deficiencies
deficiencies of
of image processing
processing
in
in automatic target recognizers
recognizers (ATRs)
(ATRs) have been independently
independently developed. Each system
exploits
exploits the
the global
global information
information in
in aa scene
scene that
that is
is discarded
discarded in
in current
current systems
systems to
to improve
upon the
upon the performance of localized ATRs. These contextual
contextual cueing
cueing systems form the initial
application
application of
of artificial technology into
into the
the automatic
automatic target
target recognizer
recognizer problem
problem domain.
domain.
Contextual target cueing is a concept in which object object classification confidence is is
enhanced by by the positive/negative
positive /negative evidence
evidence provided
provided byby their
their context. Consider the scene
shown
shown in in figure
figure 4. 4. Using
Using existing
existing statistical
statistical methods
methods, r four
four objects
objects have
have been
been identified
identified as
as
possible vehicles and have each been assigned a vehicle's confidence measurement (V1
possible (Vl=80%
=80%,r
V2=70%,
V2 =70 %, V3=80%
V3 =80%,f V4=75%).
V4 =75%). Assuming
Assuming aa threshold
threshold of
of 75%,
75%, one
one false
false vehicle
vehicle would
would be detected
detected
(V4)
(V4) while one real real vehicle would bebe omitted
omitted (V2).
(V2). By exploiting the context of of each
object, the true true confidence level level of
of each potential
potential vehicle could be determined. For For
example, VI V1 and V3 are bothboth on
on aa road,
road, in in motion and in in aa column.
column. This is is all
all positive
contextual evidence supporting aa classification
contextual classification ofof vehicle,
vehicle, so so each confidence measurement
is
is mathematically
mathematically improved
improved to
to reflect
reflect object
object context.
context. V2V2 possesses
possesses thethe same
same characteristics
characteristics
its score improves to 85%
and its 85% thus
thus exceeding
exceeding the
the 75%
75% threshold
threshold mark.
mark. V4,
V4, on
on the
the other hand
hand
is
is in
in a a lake,
lake, isolated, and has no motion. This This information is is all negative evidence
against aa classification of of vehicle but
but positive
positive evidence
evidence toward a potential secondary
classification of rock. V4's confidence drops to 60% 60% and
and is is now substantially below
threshold. After contextual analysis has completed,
After contextual completed, allall three vehicle are detected and and the
false vehicle has been discarded.
false
Two separate context cueing systems
systems have
have been
been developed.
developed. Hughes Aircraft [7] [7] has
implemented an object-oriented
object- oriented vision
vision system approach to
to automatic target recognition.
Using aa spatial
Using spatial blackboard concept
concept [8],
[8], the system maintains
maintains scene information in in a multi-
level
level inheritance
inheritance structure (Figure
(Figure 5).
5). Knowledge propagation is is the central idea
idea behind a
blackboard vision configuration. Knowledge propagation in in aa blackboard architecture
consists of two two steps. First, high confidence
confidence knowledge
knowledge sources
sources at
at all
all levels of the
identified. A propagation effect follows in which hypotheses are
blackboard are identified. are generated
for each
each high
high confidence knowledge source.
source. Hypotheses
Hypotheses are
are then
then propagated outward toward
low confidences
confidences and resolve representation
representation conflicts.
conflicts. For example, the bottom level may
For
raw image pixels, the second level
contain raw level enhanced
enhanced pixels,
pixels, the third level contrast
contrast--
derived line segments, the fourth level closed segmentation boundaries, etc.. A
etc..
hierarchical
hierarchical linking
linking of
of each
each entity (e.g. pixel
entity (e.g. (xl,yl) is
pixel (xl,yl) is part
part of
of region
region 10
10 segment 32) is
segment 32) is
maintain for
for system traceability. Information on various types of of contextual relations
internal to an
internal (such as temporal
an image (such temporal data,
data, global context, and object structure)
global context, structure) asas well
well
as
as external
external to
to the
the image (such as
image (such as TAG
TAC RECCE
RECCE and
and digital
digital terrain
terrain data)
data) can
can be
be fused into the
fused into the
blackboard
blackboard system
system to
to form
form an
an all
all encompassing
encompassing symbolic
symbolic representation
representation ofof aa scene.
scene.

SPIE Vol.528
SPIE Vol. 528 Digital
Digital Image
Image Processing
Processing(1985)
(1985)// 197
197

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/11/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

 FROM STATISTICAL CLASSIFICAT
FROM STATISTICAL ION
CLASSIFICATION RESULTS OF
RESULTS CONTEXTUAL RULE
OF CONTEXTUAL BASE PROCESSING
RULE BASE PROCESSING
V1 75%
VI VEHICLE CONFIDENCE == 75% VI
V1 75%
CONFIDENCE == 75% V2 CONFIDENCE == 80%
80%
LAKE
IN A LAKE ON A ROAD
80%
V2 VEHICLE CONFIDENCE == 80%
V2 ISOLATED IN A COLUMN
NO MOTION
NO IN MOTION
V3 70%
V3 VEHICLE CONFIDENCE == 70% 60%
NEW CONFIDENCE = 60% CONFIDENCE ==
NEW CONFIDENCE 95%
95%
80%
V4 VEHICLE CONFIDENCE == 80% 70% 80%
V3 CONFIDENCE = 70% V4 CONFIDENCE == 80%
ON A ROAD ROAD
ON A ROAD
THRESHOLD CONFIDENCE 2 75%
^ 75% IN A COLUMN IN A COLUMN
IN MOTION IN MOTION
V1,f V2
VI V4 CLASSIFIED AS VEHICLES
V2,f V4 NEW CONFIDENCE ==85%
85% 95%
NEW CONFIDENCE == 95%
V3 CLASSIFIED AS UNKNOWN
V3 V2, V3, AND
AND V4 CLASSIFIED AS
V4 CLASSIFIED VEHICLES
AS VEHICLES

4. An
Figure 4. Example Of
An Example Analysis Using
-Based Analysis
Rule-Based
Of Rule Suggestive Evidence
Using Suggestive
model objects
A knowledge base consisting of model (e.g tank, truck,
objects (e.g their
APC) and their
jeep, APC)
potential contextual relationshi ps is the focal point of
relationships of high level knowledge base
processing.
processing. Each model object is
is maintained in architecture and
in aa semantic frame architecture
the class
represents the
represents tactical objects
of tactical
class of may to
that may
objects that encountered during the processing
be encountered
to be
of a specific mission.
of models are of
mission. The knowledge base models nature similar to
hierarchical nature
of a hierarchical
the symbolic scene representat
the scene ion
representation allowing a
a hierarchica
hierarchical l classifier to
to be implemented.
The classifier
The attempts to match each unknown
classifier attempts model against known knowledge base
symbolic model
unknown symbolic
best match
models to determine aa best classification and
match classification confidences.
classification confidences.
and associated classification
Martin Marietta Aerospace [9]
Martin [9] has also implemented a model based hierarchical vision
based hierarchical
system but has taken a slightly different
different approach
approach to
to target
target recognition. Rather than
system architectur e,
integrate artificial
integrate intelligenc
artificial intelligence e at both high and
and low levels
levels in
in a
a blackboard
blackboard architecture,
Martin has separated the processes so so that they may
that they independently. The original
be worked independently.
may be
ATR has
ATR supplemented with additional algorithms
has been supplemented classification,
for global region classification,
algorithms for
motion target identificat
motion ranging, and advanced object recognition
ion, passive ranging,
identification, recognition. . When
combined with ancillary data in the form
in the of digital
form of maps, TAC
digital terrain maps, TAG RECCE, timetime-of-day
-of -day
and
and weather conditions, this information constitutes a symbolic powerful scene
representat ion which
representation is fed into an expert system context
which is context analyzer.
The context analyzer is model based
is aa model rule system capable of
based rule of reasoning about scene
context and determining true
context true object and region classification confidences through aa
region classification
positive-ne
positive- gative suggestive evidence scheme.
negative ATR data forms the to be enhanced by
classification confidence
contextual evidence classification seeds initial
confidence seeds system.
initial provided to the system. Three
types
types ofof contextual evidence exist: evidence, and neutral
evidence, positive evidence,
exist: negative evidence,
evidence.
evidence. For the fact
example, the
For example, that an object is
fact that in aa lake
is in may be
lake may for
evidence for
be negative evidence
the
the existence of
of a tank, positive evidence
evidence for
for the
the existence
existence of
of a
a boat, and neutral
evidence in
evidence in support of the object being rock.
being aa rock. The context analyzer has the the power to
feedback adjustable algorithm parameters in support of
in support hypothesis-and
of aa hypothesis- -test capability
and-test
feedback adjustable
context.
required to fully validate scene context.
required

198 //SPIE
SPIEVol. 528 Digital
Vol. 528 Processing (1985)
Image Processing
DigitalImage

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/11/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

 SHORT TERM
TERM MEMORY
MEMORY LONG TERM MEMORY
LONG TERM MEMORY

SCENE

C-1.2 41111111 ® BRIDGE

CONTEXT CNT- CLAS

BRIDGE

OBJ 1

OBJECTS OBJ-CLASSES
GRASS 4:1)
CDLID

VOLUMES VOL- CLASSES

PLANAR

SURFACES / / SRF- CLASSES

REG 10 GREEN
TRAPEZIOD

REG 11 BLUE
ECTANGLE

REGIONS REG -CLASSES

EDGES EDG- CLASSES

ACUTE r
L
J
OBTUSE

VERTICES VRT- CLASSES

PIXELS / PXL-CLASSES
PXL- CLASSES

Figure 5.
Figure 5. An
An Example
Example Of
Of The Blackboard Knowledge Representation Scheme
Scheme

SPIE Vol.
SPIE Vol.528
528Digital
DigitalImage
ImageProcessing
Processing(1985)
(1985)/ / 199
199

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/11/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

 the context
independent development of the
The independent cueing systems
context cueing led to
has led
systems has two complementary
to two
automatic target
for automatic
approaches using expert system for recognition.
target recognition. Further development of
is planned and will
each system is will concentrated on front
concentrated on extraction algorithms
image extraction
end image
front end algorithms forfor
scene analysis
scene addition to
in addition
analysis in an enhancement
to an the artificial
of the
enhancement of intelligence aspects.
artificial intelligence But
of an ATR is
improvement of
the potential performance improvement not limited to the system concepts
is not
discussed above. of reasoning
A number of exist which
techniques exist
reasoning techniques possible near
have possible
which have and
term and
near term
short term implications in
short the area
in the of target
area of recognition.
target recognition.

Summary
Artificial intelligence offers aa variety of of enhancement techniques in the area of
in the of
identified aa number
This paper has identified
computer vision. This of existing
number of existing high applications in
level applications
high level in
image understanding, knowledge base systems,
the areas of image
the knowledge
systems, inexact reasoning and knowledge
representation. in low
Applications in level processing
low level also of
using heuristics are also
processing using value.
of value.
For example, the segmentation of infrared imagery
of infrared is currently
imagery is concept in
image concept
single image
currently aa single in
which all adjustable algorithm parameters are are predetermined. By taking a multi-frame
multi -frame
approach, heuristic rules can be created to enhance the imagery and address object
obscurations and mergings during the data extraction phase ratherrather than taxing the high
than taxing
level
level symbolic processor.

200 //SPIE
SPIE Vol
Vol.528
528Digital
DigitalImage
ImageProcessing
Processing (1985)

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/11/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

 References

[1]
[1] B.
B. G.
G. Schunck,
"The Motion Constraint
"The Constraint Equation
Equation For
For Optical
OpticalFlow
Flow",
",
Seventh IJCPR, Montreal,
Seventh IJCPR, 1984.
Montreal, August 1984.
[2]
[2] C.
C. M.
M. Brown,
Brown, D.
D. H.
H. Ballard, and O.
Ballard, and 0. A. Kimball,
Kimball,
"Contraint Interaction
"Contraint InteractionInInShape-
Shape-Form-Shadowing Algorithms"
Form -Shadowing Algorithms ",,
DARPA
DARPA Image
Image Understanding Workshop, Sept.
Sept. 1982.
1982.
[3]
[3] A. R.
A. R. Bruss,
Bruss, and B.K. Horn,
Horn,
"Passive Navigation
Navigation",
",
DARPA
DARPA Image
Image Understanding Workshop, Sept 1982.
Workshop, Sept 1982.
[4]
[4] S.
S. A.
A. Shafer,
Shafer, and T.
T. Kanade,
Kanade,
"Using In Finding
"Using Shadows In Finding Surface
Surface Orientations
Orientations",
",
DARPA Image Understanding Workshop, Sept
DARPA Image Sept 1982.
1982.
[5]
[5] T. D. Garvey, J. D. Lowrance, and
J. D. and Martin
Martin A.
A. Fischler,
Fischler,
"An
"An Inference
Inference Technique for
for Integrating
Integrating Knowledge
Knowledge From
From Disparate
Disparate Sources
Sources",
",
IJCAI,
IJCAI, British
British Columbia, Canada, August 1981.
Columbia, Canada, 1981.
[6]
[6] J. F. Gilmore,
J. F.
"TESS
"TESS -The Tactical
Tactical Expert
Expert System
System",
",
Fourth International Conference On On Expert
Expert Systems,
Systems,
Avignon,
Avignon, France, 1984.
France, May 1984.
[7]
[7] Kirn,
Kim, J.
J. H.,
H., Payton, D. W.,
Payton, D. Olin, K.
W., Olin, K. E.,
E., and
and Tseng,
Tseng, D.
D. Y.,
Y.,
"A Context Dependent
Dependent Automatic
Automatic Target
Target Recognition
RecognitionSystem
System",
",
Intelligence,
Applications of Artificial Intelligence,
Proceedings
Proceedings of 485, May 1984.
of SPIE 485, 1984.
[8]
[8] Hanson,
Hanson, A.
A. R.,
R., and Riseman,
Riseman, E.
E. M.,
M.,
Computer
Computer Vision Systems,
Systems f
Press, 1978.
Academic Press, 1978.
[9]
[9] Forman,
Forman, A.
A. V.,
V., Rowland,
Rowland, P.P. J.,
J., and
and Pemberton,
Pemberton, W.
W. G.,
G.,
Analysis of
"Contextual Analysis of Tactical
Tactical Scenes
Scenes",
",
Intelligence,
Applications of Artificial Intelligence,
Proceedings
Proceedings of
of SPIE 485,
485, May 1984.
1984.
[10] D.
[10] H. Ballard, and C.
H. C. M.
M. Brown,
Computer Vision,
Englewood Cliff, Jersey,
Cliff, New Jersey,
Prentice-Hall,
Prentice -Hall, 1982.

SPIE Vol.
SPIE Vol. 528
528 Digital
Digital Image Processing
Processing(1985)
(1985)// 201
201

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 08/11/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx