Preserving and Applying

Human Expertise:
Knowledge-Based Systems
Chapter Objectives
‰ KB systems are adept at preserving captured
and/or discovered knowledge for later sharing
and/or application.
‰ Introduce the internal operation of knowledge-
based systems, including:
„ Knowledge representation
„ Automated reasoning
‰ Introduce the art of knowledge engineering - how
to develop knowledge-based systems
„ the tools
„ the techniques
Objectives

‰ Object-oriented representations (frames, classes,

UML)
‰ Class inheritance hierarchies
‰ Logical databases (relational and object-
relational)
„ Inference engines and shells
„ Knowledge engineering
„ Forward vs backward reasoning
„ Default-based reasoning using frames
Objectives
„ KBS from the points of view of those that
work with them:
‰ The user
‰ The knowledge engineer
„ Introduce the different components of a KBS
‰ The inference engine
‰ The knowledge base
‰ The user interface
‰ The fact base
‰ The development environment
KBS- the end user’s point of view

„ From end-user’s perspective, KB system has

three components:
Intelligent program
User interface
Problem-specific database (“workspace”)
User’s view of a KBS

Intelligent
Program
User
Interface
User

Workspace
The intelligent program

„ A black box, to the end user, core of KBS

„ Encapsulates most of the knowledge,
including possibly knowledge representations
of rules, frames, defaults, is-a and has-a
hierarchies, etc.
„ Typically rarely necessary for end users to
access, and often dangerous from a security
standpoint
User interface
„ May enable the intelligent program to pose
questions to the user about the problem at hand
„ May provide explanations about why the intelligent
program is asking particular questions
„ May allow the user to query the intelligent program
as to why or how a particular decision was made
„ Displays results
„ May provide graphic representations of the results
(e.g., decision tree paths, parts of the instantiated
class hierarchy, etc.)
„ May allow the user to save or print results
Workspace
„ Problem-specific database (“workspace”)
„ “Database” in this context refers not to some flat or
relational database, but rather a logical
representation of known facts (beginning with
axioms, from which inferences are drawn)
„ The working space where the system reads any
inputs and writes its outputs (in logic form)
„ The inputs consist of all information provided either
automatically or by the user via the UI
„ The outputs consist of all conclusions the intelligent
program is able to drive, including both the final
solution required by the user, as well as
intermediate conclusions that act as steppingstones
in the path to the final conclusion
KBS- the developer’s point of view
Developer of the core platforms or languages
„ develop “shells”, inference engines, UIs, etc.
„ not usually necessary; just use one or more of the
many existing
„ development environments
Knowledge engineer (KE)
„ use the development environment (e.g., rule
language, object oriented language, etc.) to
“program” a KB system
„ traditionally, personally interview domain experts to
elicit knowledge
„ modern automated tools can assist, and supplement
face-to-face interviews
KBS-developer’s view

„ KBS consists of two major components:

‰ Intelligent program
‰ Development environment
KE’s view of a KBS

Development Intelligent
Shell Program

Knowledge Knowledge
Acquisition base
Tool

Test Case
Database
Inference
Engine
Developer’s
Interface
The Intelligent Program
„ Unlike the end user, the KE can open the
black box.
„ Recall key differences of KB systems vs.
conventional software:
‰ The use of highly specific domain knowledge.
‰ The heuristic nature of the knowledge employed,
instead of exact.
‰ The separation of the knowledge from how it is
used.
Inside the black box

„ Knowledge base (the knowledge; declarative)

„ Inference engine (how it is used; procedural)
KB
‰ Arguably the most important component of a KB
system
‰ Contains the entire relevant, domain-specific,
problem-solving knowledge gathered and
integrated by KEs from the various available
sources
‰ The KB is the knowledge to be managed.
Representing the Knowledge Base
„ How to represent the knowledge in a KB depends on its nature:
Functional
„ in the realm of traditional mathematical functions
„ addressed by conventional programming techniques
Heuristic
„ most KB applications occur in domains where conventional
computational problem-solving approaches either don’t exist or don’t
work well
„ thus the knowledge consists of heuristics – rules of thumb and
shortcuts learned and developed by experts over years of practical
problem solving experience
„ often naturally expressed as rules (IF-THEN statements)
if the house plan includes a swimming pool, then add $100K to price
Structured
„ where knowledge expresses the association of several components
(parts) that, when composed, form a larger assembly (whole)
„ often naturally expressed as frames (objects)
Intelligent Program –the inference engine

„ The inference engine is the interpreter of the

knowledge stored in the knowledge base
„ It examines the contents of the KB and the
input data about a problem
„ Exercises the knowledge to derive the
conclusions or answer the questions asked.
„ The KB can be thought as a very large graph.
Problem description graph- heuristic
system to classify fruits
Graph
„ Nodes on left side: represent all the signs, symptoms,
characteristics, and features used as inputs when attempting to
solve a relevant problem
„ Nodes on right side: represent all possible solutions to the
problem
„ Nodes in middle: represent intermediate conclusions, which the
system derives in the progression of deriving the final solution(s)
„ Edges: represent the knowledge used by the expert during
problem solving (elicited from experts and stored in the KB)
„ node on its left end: an antecedent of an IF-THEN rule
„ node on its right end: the consequent of the same IF-THEN
rule
„ Short arcs linking some edges: represent a logical AND of the
antecedents on the edges’ left ends, in the IF-THEN rule
What the inference engine does

„ The job of the inference engine can be thought of

as:
‰ exploring this huge graph (which typically can be
exponentially large)
‰ finding path(s) connecting the nodes on the right to the left
(inputs and solutions)
‰ If we work from the inputs to the solutions (forward
reasoning)
‰ If we work from the solutions to the inputs (backward
reasoning)
‰ From both ends simultaneously (bidirectional reasoning)
The Developer’s environment

„ This is a toolkit facilitating the creation of

knowledge within the intelligent program
„ It assists the developer with structuring,
debugging, modifying, and expanding the
“program” which in this case is knowledge
gathered from experts
‰ Knowledge acquisition tool
‰ Test case database
‰ Developer’s interface
Developer’s view

Development Intelligent
Shell Program

Knowledge Knowledge
Acquisition base
Tool

Test Case
Database
Inference
Engine
Developer’s
Interface
Knowledge acquisition tool
„ Assists the KE in the construction of the KB
Simplest form:
„ Serves as a KB editor, providing a view of the knowledge (rules,
frames, etc) and offering editing functions
„ Sophisticated form can add a wide range of features:
‰ Multiple browsers for the KB, in both text and graphical forms,
hierarchical as well as linear
‰ Debuggers and checkers to assist KE in locating “bugs” (logical
inconsistencies, typos, etc)
‰ Tools that compare existing knowledge to new knowledge, to
guess what the KE really means in case the KE is not as precise
as required
‰ Version control mechanisms to provide fine-grained bookkeeping
Test Case Database

„ The KE often makes potentially significant

(and dangerous) changes to the KB.
‰ Deleting existing knowledge could accidentally
eliminate important relationships, preventing
needed inferences
‰ Modifying existing knowledge could also
accidentally change important relationships
‰ Adding new knowledge could accidentally
introduce contradictions with existing knowledge,
or even reprioritize inferences such that the
desired conclusions are no longer drawn
Test case database

„ Any of these could compromise the

accuracy/integrity of the knowledge base –
checks are essential.
„ Many KB systems include a test case
database
‰ Sample problems that have been successfully
executed on the system in the past
‰ Whenever the KB is edited, the test cases are all
re-executed to verify that these benchmark cases
are still solved correctly.
Developer’s interface

„ Usually, an augmented version of the UI seen

by the end user
„ Allows the KE to exercise the KB as it is
modified and retested
„ Permits the KE to see exactly how the
system can operate when delivered to the
end user
„ When combined with KA tool, powerful
environment for developing a KBS
Representing Knowledde

„ Introduce the means of manipulating the

knowledge found in a knowledge base.
„ Reasoning with frames
„ Reasoning with rules
‰ Forward reasoning
‰ Backward reasoning
Representing knowledge: rules/1

„ Heuristic knowledge is often nicely

represented as rules.
‰ IF-THEN rules are highly intuitive for both KEs
and experts to think about, and to understand
once implemented.
‰ If you ask skilled experts about how they solve
problems or how they reached a certain
conclusion, they typically respond with knowledge
expressed in a rule format
Rules/2

„ Well, I noticed that A, B, and C were present in this

problem, and these three facts imply that D is true.
IF A, B, C THEN D
A∧B∧C⇒D
„ A, B, C are the antecedents (or premises or
conditions/situations)
„ D is the consequent (or conclusion or action)

„ A big drawback of rules: KEs have trouble

maintaining consistency in a rule base when number
of interacting rules becomes too large
Rules/3

„ Rules can be used to express a wide range

of associations
„ They can express actions and situations that
must be taken
‰ If you are driving a car and an emergency vehicle
approaches, then you should slow down and pull
to the side of the road to allow the emergency
vehicle to pass
‰ If baking a cake, test for completion by inserting a
toothpick in the cake’s centre. If it emerges clean,
the cake is ready to be taken out of the oven
Rules/4

„ Rules can express premises and conclusions

that can be drawn from those premises
(diagnosis/explanation)
‰ If your body temperature is above 37°C, then you
have a fever.
‰ If the outside temperature is below freezing, the
gas gauge on your car does not register empty,
and the engine turns over but will not start, then it
is highly likely that you have a frozen gas line.
Rules/5

„ Can express antecedents and their

consequences (cause-and-effect)
‰ If you don’t read the textbook or attend class, then
you will flunk the exam.
‰ If the tub’s drain is clogged and the water is left
running, then the floor will become wet.
‰ If the electric bill is past due and your credit rating
is bad, then your electricity will be cut off.
Inference chains

„ With pattern matching, rule-based systems

use automated reasoning methods to
progress logically from data to conclusions.
„ Process of problem-solving in KB systems is
to create a series of inferences that form a
“path” between the problem definition and its
solution.
„ Such a series of inferences is called an
inference chain.
Examples of inference chains
Rule 1
IF the ambient temperature is above 27C
THEN the weather is hot

Rule 2
IF the relative humidity is greater than 70%
THEN the atmosphere is humid

Rule 3
IF the weather is hot and atm. Humid
THEN thunderstorms are likely to develop
Inference chains

„ Inputs
Fact 1 Æ the ambient temperature is 26C
Fact 2 Æ the relative humidity is 70%
„ Facts are statements considered true within
the KBS and found in the workspace
„ We can now deduce the following new facts
Chain of rules

FROM
Fact 1
„ the ambient temperature is 26C
Rule 1
IF the ambient temperature is above 26C
THEN the weather is hot
„ The inference engine can deduce:
Fact 3
the weather is hot
Chain of rules

From:
Fact 1
„ the relative humidity is 70%
Rule 2
„ IF the relative humidity is greater than 65%
„ THEN the atmosphere is humid
The inference engine can deduce:
Fact 4
„ the atmosphere is humid
Chain of rules
From:
Fact 3
„ the weather is hot

Fact 4
„ the atmosphere is humid

Rule 3
„ IF the weather is hot and the atmosphere is humid

„ THEN thunderstorms are likely to develop

The inference engine can deduce:

Fact 5
„ thunderstorms are likely to develop
Reasoning with uncertainty

„ Note the use of “likely” in “thunderstorms are

likely to develop”
„ Ideally we’d like a more precise prediction
than “likely”
„ Modern systems provide mechanisms for
reasoning under uncertainty
‰ Probability (especially Bayesian methods)
‰ Fuzzy logic
Frames/1

„ Rules do not provide a natural representation

for:
‰ grouping facts into associated clusters
‰ associating relevant procedural knowledge with
some fact or group of facts
„ Frames were developed by Marvin Minsky
[1975] for this purpose, in his landmark paper
“A Framework for Representing Knowledge”
http://web.media.mit.edu/~minsky/papers/Frames/frames.html
Minsky frames/2
„ We can think of a frame as a network of nodes and relations.
„ The "top levels" of a frame are fixed, and represent things that
are always true about the supposed situation.
„ The lower levels have many terminals "slots" that must be filled
by specific instances or data.
„ Each terminal can specify conditions its assignments must meet.
(The assignments themselves are usually smaller "sub-frames.")
„ Simple conditions are specified by markers that might require a
terminal assignment to be a person, an object of sufficient value,
or a pointer to a sub-frame of a certain type.
„ More complex conditions can specify relations among the things
assigned to several terminals.
Frames/3

„ A frame provides the structure or framework

for representing structured knowledge
„ Using a structure consisting of a frame name
and a set of attribute-value pairs, a frame
represents a stereotypical situation (i.e., an
object, concept, or process) from the real
world
„ Frames are similar to database records
Basic Frame for Mustang
Frame: MUSTANG
Manufacturer: Ford
Country of Manufacture: USA
Model: Mustang GT
Number of wheels: 4
Number of doors: 2
Year: 2002
Engine Size: 4.6L-V8
Transmission: Standard
Reliability: Medium
Body Style: Convertible
Color: Red
Miles-per-gallon: 19.7
Serial Number: 12345A67890B
Owner: Avelino
Details about Frames

„ Use general attributes that do not change

„ Inheritance of information, no need to repeat
information
„ Additionally, functional attributes (whose values are
executable functions) can add representational
power to frames
„ Also known as methods (“member functions” in
C++)
‰ if we wish to know the reliability of 2002 Mustangs based
on the fleets’ experience, this number could change
constantly. A method could provide more reliable up-to-
date numbers than storing a fixed value.
Attributes

„ Attributes are also known as slots

„ Values are also known as fillers
‰ Each attribute/slot/member can be filled by a
value/filler
‰ the fillers can be subdivided into facets each
having its own associated value
‰ The VALUE facet is the most important
„ It contains the actual value assigned to that slot (like
holding the value of a variable)
Attributes in more detail
„ Several other facets besides VALUE:facet can be
associated with each slot:
‰ Range: the range of possible values for this slot
‰ Legal-values: the set of discrete possible values the slot
can take (like enums in C++)
‰ Default: the value to assume if none is explicitly stated
‰ If-needed: Procedure(s) for determining the actual value
‰ If-added: Procedures to execute when a value is specified
for the slot
‰ If-changed: Procedures to execute if the value of the slot is
changed
„ The last three, all procedures, are also called
daemons
Generic frame for an automobile
G e n e ric A U T O M O B IL E F ra m e
S p e c ia liz a tio n – o f: V E H IC L E
G e n e ra liza tio n – o f: (S T A T IO N – W A G O N C O U P E S E D A N C O N V E R T IB L E )
M a n u fa ctu re r:
V a lu e :
L e g a l-v a lu e s : (F O R D M A Z D A B M W S A A B G M C H R Y S L E R )
D e f a u lt: F O R D
C o u n try – o f – M a n uf a ctu re :
V a lu e :
L e g a l-v a lu e s : (U S A J A P A N G E R M A N Y S W E D E N )
If -N e e d e d : (G E T -O R IG IN )
D e f a u lt: U S A
M o d e l:
V a lu e :
L e g a l-v a lu e s : (T A U R U S F O C U S M U S T A N G C R O W N -V IC T O R IA ))
C o lo r:
V a lu e :
L e g a l-v a lu e s : (B L A C K W H IT E R E D P E R S IA N – A Q U A )
If – N e e d e d : (E X A M IN E – T IT L E o r C O N S U L T – D E A L E R o r L O O K –
A T – A U T O M O B IL E )
R e l i a b i lit y :
V a lu e :
L e g a l-v a lu e s : (H IG H M E D IU M L O W )
D e f a u lt: M E D IU M
M il e s – P e r – G a ll o n :
V a lu e :
R a n g e : (0 – 1 0 0 )
Y e a r:
V a lu e :
R a n g e : (1 9 4 0 – 1 9 9 0 )
If – C h a n g e d : (E R R O R : V a lu e c a n n o t b e m o d ifie d )
O w n e r:
V a lu e :
If – A d d e d : (A P P L Y – F O R – T IT L E a n d O B T A IN – T A G a n d P A Y –
S ALE S–TA X)
Inheritance

„ To avoid including, for example, the number

of wheels in every frame for every car, we
use inheritance
‰ All attributes that do not change (the general
attributes) are moved up to a higher level frame.
‰ The more specific frames inherit that information.
Defining new frames is easy with
inheritance

Generic COUPE Frame

Specialization–of: AUTOMOBILE
Generalization–of: (SUZIE-SMITH’S-AUTOMOBILE, JOHN-DOE’S-
AUTOMOBILE AVELINO’S-AUTOMOBILE)
Doors:
Value: 2
The inheritance hierarchy

VEHICLE Generic frame

AUTOMOBILE Generic frame

COUPE Generic frame

MUSTANG Generic frame

AVELINO’S- Instance-level frame

AUTOMOBILE (Describes a real automobile)
Generic frame Mustang

Generic MUSTANG Frame

Specialization-of: Coupe
Generalization-of: AVELINO’S-AUTOMOBILE
Manufacturer:
Value: FORD
Engine:
VALUE:
LEGAL-VALUES: (3.0L-V6, 4.6L-V8)
Specific frame for an automobile
AVELINO’S-AUTOMOBILE Frame
Specialization–of: Mustang
Manufacturer:
Value: Ford
Vehicle ID No.:
Value: 12345AJG67890
Country–Of–Manufacture:
Value:
Color:
Value: Red
Reliability:
Value: Medium
Miles–Per–Gallon:
Value: 19.7
Year:
Value: 2002
Owner:
Value: AVELINO
Doors:
Value:
Automated Reasoning Process

„ Introduce the means of manipulating the

knowledge found in a knowledge base.
„ Two main KR paradigms, two types of
reasoning
‰ Reasoning with frames
‰ Reasoning with rules
„ Forward reasoning
„ Backward reasoning
Frame based reasoning

„ Modern knowledge representations differentiate

between terminological and assertional knowledge
[KL-ONE]
Terminological
„ “a red car belonging to Bob”
‰ is just a term that we can use
‰ does not say anything about the real world
‰ in fact, there might not even exist any red car belonging to
Bob in the real world
„ Reasoning: respond to queries about the definitions
of various concepts (classes/categories,
objects/instances)
Frame based Reasoning

„ Assertional
‰ “the car belonging to Bob is red”
‰ “the red car belongs to Bob”
‰ “a red car belonging to Bob does not exist”
„ Reasoning: respond to queries about the
known state of the world
Frame-based reasoning
„ Excels at defining terms (i.e., for representing
terminological knowledge)
„ Is generally used as support for defining
terms or storing concepts, within more
complex systems that use other mechanisms
for reasoning about assertional knowledge
Rule based Reasoning
„ The objective of a reasoning process is to derive a value for
a conclusion.
„ But there is a wide variation of ways this can be done.
‰ In many situations, humans find it most natural to progress from
the initial data to a final answer.
‰ E.g. thunderstorm example earlier.

‰ Makes good sense whenever

„ relatively few input data are required, or
„ there are many possible conclusions.
„ Alternatively, when there is much data but only a small portion is
relevant, considering all the data would be highly inefficient.
‰ you only tell the doctor your abnormal symptoms (headache,
fever) and not all the other things that are okay (neck doesn’t
hurt, back feels fine
‰ Doctor tries to determine the most likely diagnosis based on the
limited input data, and then prove the hypothesis via additional
questions or tests.
Forward vs Backward Reasoning

„ So there are two means of deriving

conclusions:
„ Start with all the known data and progress
toward the conclusion – data driven, forward
chaining, or forward reasoning
„ Select a possible conclusion and try to prove
its validity by looking for supporting evidence
– goal driven, backward chaining, or
backward reasoning
Rule based reasoning
Rule based reasoning
Rule 1
IF Surface = smooth
AND Fruitclass = vine
AND Color = green
THEN Fruit = honeydew
„ Each rule describes some characteristic of the different fruits
through a series of parameters, eg:
‰ Fruit
‰ Fruitclass
‰ Seedclass

„ Parameters that represent the final answer are conclusions or

goals.
‰ Fruit
„ The others are called intermediate
parameters/conclusions/goals.
„ A combination of parameter and value that is
considered true is called a fact.
„ The workspace (or database) contains all the facts
known to the system at any given point in time
during the execution of the KB system.
„ By assigning the goal a value (eg, Fruit = apple), the
system solves the problem of classifying a specific
instance of a fruit.
Forward chaining

„ Moves from the input data toward the

conclusions/goals.
„ Inputs are used to satisfy the premises of applicable
rules, which allows them to be executed (“fired”)
thus setting values of other (intermediate or final)
parameters, yielding newly derived facts.
„ These new facts intern may cause the premises of
other rules to be satisfied and fired.
„ This process is called rule interpretation.
Forward chaining-rule interpretation
process

Facts

Facts
New Facts

Step 1 – Step 2 – Step 3 -

Conflict Execution
Match Resolution
Applicable Selected
Rules Rule

Knowledge
Rules New
Rules
Rule interpretation steps

1. Matching: rules in KB compared with known facts

to decide which rules are satisfied
2. Conflict resolution: if multiple rules are satisfied,
select rule with highest priority from potentially
executable rules
3. Execution: last step in interpretation (firing of
rules). This execution has one of 2 possible
outcomes
1. A new facts (or facts) can be derived, added to workspace
2. Or a new rule (or rules) added to set of rules (KB)
Trace of rule-based execution

Execution Applicable Selected Derived

Cycle rules rule Fact
1 3,4 3 Fruitclass = tree
2 3,4 4 Seedclass = stonefruit
3 3,4, 11 11 Fruit = cherry
4 3, 4, 11 — —
Forward chaining

„ Best suited for problem domains involving

synthesis:
‰ design
‰ configuration
‰ planning
‰ control
‰ scheduling
„ Domains where the data drive the solution
approach
Backward chaining

„ Moves from a hypothesized goal backward toward

the input data, looking for some way to assemble
enough supporting evidence from the input data.
‰ start with a single top-level goal in our fruit classification
example – the goal is indicated by the goal parameter Fruit
„ Backward inference can also already begin without
any inputs specified
‰ Inputs can be requested from the user or otherwise
acquired, only as they become necessary to derive a value
for the selected goal.
Backward chaining

„ When an answer is found, execution can

stop, or it can go on to attempt to satisfy
another goal.
‰ This process is called tracing a goal.
„ The rule interpreter for backward reasoning
differs significantly from that of forward
reasoning.
‰ Typically, the workspace is initially empty.
Rule interpretation process of backward
chaining
1. Form a list initially composed of all top-level
goals defined in the system. The top-level
goals are predefined by the developer.
2. Consider the first goal from the list. Gather
all rules capable of deriving a value for this
goal.
3. For each of these rules, in turn examine its
premises:
1. If all its premises are satisfied (ie, each premise has its specified
value contained as a fact in the workspace), then execute this rule
to derive its conclusions. Remove this goal from the list and return
to step 2. This is done because a value has been derived for the
current goal.
2. If a premise of a rule is not satisfied because of the absence of a fact
to satisfy one of its premises (i.e., one of the premise’s values does
not exist as a fact in the workspace), look for rules that derive the
specified value for this premise. If any can be found, then consider
this premise to be a subgoal, place it on the beginning of the goal
list, and go back to step 2.
3. If step b cannot find a rule to derive the specified value for the current
premise, then query the user for its value and add it to the
workspace as a fact. If this value satisfies the current premise, then
continue with this rule’s next premise. If the premise is not satisfied,
then consider the next rule.
„ If all rules that can satisfy the current goal
have been attempted and all have failed to
derive a value, then this goal remains
undetermined. Remove it from the list and go
back to step 2. If the list is empty (ie, all top-
level goals have been processed), then halt
and announce completion.
Backward chaining

„ Rule-based systems generally follow the

closed world assumption:
‰ If a fact specifically asserting something is not
present, then assume it to be false.
„ Thus backward chaining along a particular
path will fail, if it requires some fact that
cannot be proved from any input facts.
„ Best suited for diagnostic problems that have
a relatively small number of possible
conclusions
Problems with developing KBS

„ Requirements are harder to define than for

standard software
„ Human expertise is difficult to define and
even more difficult to elicit
„ Unlike standard software, requires developer
to maintain close continuous contact with
experts throughout the entire development
process
„ Desirable qualities of knowledge engineers can be
more challenging than for standard software
engineers
„ Must depend equally on intuition and personal
qualities to be successful, eg
‰ ability to absorb broad and deep knowledge in non-CS
domains
‰ the ability to get along well with others
‰ Must still have good “programming” (logic, design,
organization, documentation, debugging, and testing) skills
just like other SEs
Conclusions

‰ Rules and how they are used to represent and

exercise conditional knowledge.
‰ Frames and how they are used and exercise
structured knowledge.