Knowledge Engineering

(IT4362)

Quang Nhat Nguyen

([email protected])

Hanoi University of Science and Technology

School of Information and Communication Technology
Academic year 2021-2022
Content
◼ Introduction
◼ First-order logic
◼ Knowledge representation
• Semantic networks
• Inheritance
◼ Logic programming
◼ Expert systems
◼ Uncertain reasoning
◼ Knowledge discovery by Machine learning
◼ Knowledge discovery by Data mining

Knowledge Engineering 2
Semantic networks – Motivation
◼ SN was first proposed by Quillian in 1966, as a model of
human memory
◼ Motivations
❑ To understand the structure of human memory, and its use in language
understanding
❑ What sort of representational format can permit the “meanings” of words
to be stored, so that humanlike use of these meanings is possible?
❑ Claim that people use same memory structure for a variety of tasks

◼ Psychological evidence that memory uses associative links in

understanding words
◼ Wish to encode dictionary definition of words, in order to
❑ Compare and contrast meanings of two words
❑ Generate quasi-English sentences to describe the comparison
Knowledge Engineering 3
Semantic networks – Introduction
◼ Semantic network (SN) is a graph-based representation

◼ A SN is a network of nodes and links to represent the

definition of a concept (or a collection of concepts)
❑ The nodes represent concepts
❑ The links represent the relations between concepts

◼ The reasoning (inference) in SNs is done based on the

propagation of
❑ Activation
❑ Inheritance

Knowledge Engineering 4
Semantic networks – Syntax
◼ Nodes represent concepts, actions or objects in the
problem domain
◼ Links are directional and labeled relations between nodes
◼ Two kinds of links: inheritance-oriented and domain-
specific
◼ Inheritance-oriented link, to represent that:
❑ Node A is a sub-class (sub-category) of node B (i.e., IS-A link)
❑ Node A is an instance of node B (i.e., INSTANCE-OF link)

◼ Domain-specific link, to represent that:

❑ Node A relates to (i.e., has a relation with) node B
❑ E.g., HAS, CAN, HAS-PART, CAUSES, HAS-COLOR, … links
Knowledge Engineering 5
Semantic network – Example (1)

(B. L. Vrusias, course AI-CS289)

Knowledge Engineering 6
 Semantic networks – Example (2)
c o v e re d _ b y
a n im a l
tr a v e ls _ b y s k in
fly in g
is a is a
tr a v e ls _ b y
b ir d fis h
fe a th e rs
c o v e re d _ b y s w im m in g

is a is a is a is a

o s tr ic h p e n g u in c a n a ry r o b in

tr a v e ls _ b y c o lo u r
c o lo u r

w a lk in g y e llo w
tr a v e ls _ b y

re d

in s ta n c e _ o f

in s ta n c e _ o f
O pus

T w e e ty

c o lo u r
w h ite
(http://www.cwa.mdx.ac.uk/bis2040/johnlect.html)

Knowledge Engineering 7
SNs – Spreading activation
◼ Given two concepts, Living Thing
spreading activation will is-a
activate concepts by
Animal is-a
following links from one or
both
is-a
eats Plant
◼ Allows identification of
concepts ”between” these Elephant
is-a
two concepts that help
relate them is-a Food
❑ E.g., Spreading activation
between the two concept Mammoth
“Elephant” and “Plant”

Knowledge Engineering 8
Semantic networks – Inheritance
◼ Properties of super-classes Legs
(super-categories) are inherited
to sub-classes (sub-categories) has-part

Mammal
◼ Universal inheritance: All
Grey
relations are inherited is-a
has-color
◼ Default inheritance: Relations
are inherited unless there are Elephant
conflicting information at a more White
specific node is-a
has-color
◼ Psychological studies shows that Clyde
in humans response time:
“Clyde has-color White” < “Clyde
has-part Legs”
Knowledge Engineering 9
Semantic networks – Semantics (1)
◼ Semantic networks appear intuitive – The syntax is
seductively uniform
◼ But: Different systems representing the same “graph”
have different interpretations
◼ What are the semantics of Semantic Networks?
❑ ”Since the semantics of any given language is dependent of the
interpretation of the primitive elements....., the well-definedness of
a network language rests heavily on the set of node and link types
that it provides” (Brachman, p204, Readings in KR)

Knowledge Engineering 10
Semantic networks – Semantics (2)
◼ What does the following SN mean?
❑ This is a definition of a “black- Car
colored” car (Definitional
information) is-a
❑ This states that there exists a car
that is black (Assertional
Car#3
information)
❑ This states that a particular car has-color
(i.e., car#3) is black (Asserts
existence)
Black

Knowledge Engineering 11
Semantic networks – Semantics (3)
◼ Links can be...
❑ Assertional
◼ Captures the information about the world
◼ May change when the world changes
◼ E.g., “Jon hit Mary” (a concrete event that happened)

❑ Definitional
◼ Captures the meaning of concepts
◼ Does not change when the world changes
◼ E.g., “apple is-a fruit”, “apple has-color red”

Knowledge Engineering 12
SNs – Transformation to Logic (1)
◼ Possible solution to the semantics problem of SNs:
Transform SNs to logic
◼ The idea: Logic has well-defined semantics, grounding in
logic will provide well-defined semantics for SNs
◼ Are the nodes simply constants and the relations binary
predicates?
❑ Not quite so simple – But the syntax can be rewritten in logic

Knowledge Engineering 13
SNs – Transformation to logic (2)
◼ For is-a links
is-a
❑ x: elephant(x) → mammal(x) Elephant Mammal

◼ For links from instances

has-color
❑ hasColor(Clyde, White) Clyde White

◼ For links from classes

(categories) has-color
Elephant Grey
❑ x: elephant(x) → hasColor(x, Grey)

◼ Consider what the reasoning of logic

will conclude if we assert all these
sentences + “elephant(Clyde)”?
❑ Default reasoning breaks!
Knowledge Engineering 14
Semantic networks – Advantages
◼ Explicit in visualization and easy to understand
❑ Often used as a communication tool between the knowledge
engineer and the expert during the knowledge acquisition phase

◼ SNs are particularly good at representing knowledge in

the form of hierarchies
❑ Knowledge is hierarchically categorized (classified)

◼ The net is its own index – Quick inference possible

◼ Supports default reasoning in finite time

Knowledge Engineering 15
Semantic networks – Disadvantages
◼ No interpretation standard – Lack well-defined semantics
◼ Problematic in representation of negation and disjunction
❑ E.g., “John does not go fishing”, “John eats pizza or fish and
chips”

◼ Hard to choose good primitives

◼ Quite limited inference possible

Knowledge Engineering 16
Inheritance – Hierarchical taxonomy
◼ Hierarchical taxonomy is a natural way to view the world
❑ Used in frames and in semantic networks

◼ The importance of abstraction in remembering and

reasoning
❑ Groups of things share properties in the world
❑ Don’t have to repeat representations

◼ Inheritance is the result of reasoning over paths in a

hierarchy
❑ “Does A inherit from B?” is the same as “Is B in the transitive
closure of is-a (or subsumption) from A?”

Knowledge Engineering 17
Graphical representation of inheritance
◼ IS-A relations
❑ Clyde is an Elephant, Elephant is Gray Gray

◼ Reasoning with paths and

conclusions they represent
❑ Transitive relations Elephant

◼ Transitive closure
❑ Clyde is Elephant, Elephant is Gray,
Clyde is Gray Clyde

Knowledge Engineering 18
Inheritance networks (1)
◼ Tree structures with
strict inheritance Gray

❑ Conclusions produced by
complete transitive closure
on all paths
Rat Elephant
❑ All reachable nodes are
implied

Ben Clyde

Knowledge Engineering 19
Inheritance networks (2)
◼ Lattice structures
with strict Illiterate Tax-payer Salaried
inheritance Academic
❑ With multiple AND
parents (i.e.,
Student Employee
multiple inheritance)

◼ Same as in tree
structures: All
Ernest
conclusions you can (Represents
reach by any paths “is-not”)
are supported

Knowledge Engineering 20
Inheritance networks (3)
◼ Ambiguity in inheritance
Pacifist
◼ Example
❑ Is Nixon a pacifist or not?
Quaker Republican

Nixon

Knowledge Engineering 21
Inheritance networks (4)
◼ Defeasible inheritance
Gray
❑ As in frame systems
❑ Inherited properties do not
always hold, and can be
Elephant
overridden (defeated)
❑ Conclusions determined
by searching upward from
“focus node” and selecting Clyde
first version of property
you want Elephant is Gray,
but Clyde is not

Knowledge Engineering 22
Inheritance networks (5)
◼ Defeasible inheritance Aquatic creature Gray
❑ Links have polarity
(positive or negative)
❑ Use the shortest path Mammal Elephant
heuristic to determine
which polarity counts
❑ As a result, not all paths
count in generating Whale Royal elephant
conclusions – Some are
“pre-empted” but some
White whale Fat royal elephant
are “admissible”
❑ Think of paths as
arguments in support of Baby Beluga Clyde
conclusions

Knowledge Engineering 23
Inheritance hierarchy – Formal def. (1)
◼ An inheritance hierarchy G = <V,E> is a directed and
acyclic graph (DAG) with positive and negative edges,
intended to denote “(normally) is-a” and “(normally) is-
not-a”, respectively
❑ Positive edges are written a • x
❑ Negative edges are written a • x

◼ A sequence of edges is a path

❑ A positive path is a sequence of one or more positive edges:
a•…•x
❑ A negative path is a sequence of positive edges followed by a
single negative edge: a • … • v • x

Knowledge Engineering 24
Inheritance hierarchy – Formal def. (2)
◼ There are no paths with more than 1 negative edge
◼ There might be 0 positive edges
◼ A path (i.e., argument) supports a conclusion
❑ Path a • … • x supports the conclusion “a is an x”
❑ Path a • … • v • x supports the conclusion “a is not an x”

◼ A conclusion may be supported by many arguments (i.e.,

by many paths)
◼ However, not all arguments (i.e., all paths) are equally
believable

Knowledge Engineering 25
Support and Admissibility (1)
◼ The hierarchy G supports a conclusion “a is x” if it
supports some corresponding path
❑ Similar definition for G support a conclusion “a is not x”

◼ The hierarchy G supports a path a • s1 • … • sn • x if

the corresponding set of edges {a • s1 • … • sn • x } is in
E, and the path is admissible
❑ Similar definition for G support a path a • s1 • … • sn • x

◼ A path is admissible if every edge in it is admissible

Knowledge Engineering 26
Support and Admissibility (2)
◼ A positive edge v • x is admissible in G w.r.t. node
a if there is a positive path a • s1 • … • sn • v (n ≥ 0) in
E and:
1. Each edge in a • s1 • … • sn • v is admissible in G w.r.t. node
a (recursively);
2. No edge in a • s1 • … • sn • v is redundant in G w.r.t. node a
(“redundant” is defined in the next slide);
3. No intermediate node a, s1, …, sn is a pre-emptor of v • x
w.r.t. node a (“pre-emptor” is defined in the next slide)

◼ Similar definition for a negative edge v • x is

admissible

Knowledge Engineering 27
Redundancy
◼ A positive edge b • w is
redundant in G w.r.t. node a if Mammal (w)
there is some positive path (b • t1 •
… • tm • w)  E (m ≥ 1), for which b•w

1. Each edge in b • t1 • … • tm is Whale (t1)

admissible in G w.r.t. node a;
2. There are no c and ti such that c • ti
is admissible in G w.r.t. node a; Blue whale (b)

3. There is no c such that c • w is

admissible in G w.r.t. node a Whale-01 (a)

◼ Similar definition for a negative

edge b • w is redundant

Knowledge Engineering 28
Pre-emptor
◼ A node y along path a • ... • y •
... • v is a pre-emptor of Aquatic creature (x)
positive edge v • x w.r.t. node a
y•x
if (y • x)  E
Mammal (v)
◼ A node y along path a • ... • y •
... • v is a pre-emptor of
negative edge v • x w.r.t. node Whale (y)
a if (y • x)  E
❑ See an example in the figure on the Blue whale (a)
right!

Knowledge Engineering 29
Credulous extensions
◼ G is a-connected if and only if for every node x in G
there is a path from a to x, and for every (negative)
positive edge v • ()x in G there is a positive path from
a to v
❑ In other words, every node and edge is reachable from a

◼ G is (potentially) ambiguous w.r.t. a node a if there is

some node x  V such that both (a • s1 • … • sn • x)
and (a • t1 • … • tm • x) are paths in G
◼ A credulous extension of G w.r.t. node a is a maximal
unambiguous a-connected sub-hierarchy of G w.r.t.
node a

Knowledge Engineering 30
Credulous extensions – Example
Milk- Milk-
producer producer

Mammal Mammal Mammal

Furry Egg- Furry Egg- Furry Egg-

Animal layer Animal layer Animal layer

Platypus Platypus Platypus

Extension 1 Extension 2

Knowledge Engineering 31
Preferred extensions
◼ Credulous extensions do not incorporate any notion of
admissibility or pre-emption

◼ Let X and Y be credulous extensions of G w.r.t. node a.

X is preferred to Y if and only if there are nodes v and
x such that:
❑ X and Y agree on all edges whose endpoints precede v
topologically,
❑ There is an edge v • x (or v • x) that is inadmissible in G,
❑ This edge is in Y, but not in X

Knowledge Engineering 32
Preferred extensions – Example
Aquatic creature (x)

Preferred
Aquatic creature (x) Mammal (v)

Whale (a)
Mammal (v)

Whale (a) Aquatic creature (x)

Mammal (v)

Whale (a)

Knowledge Engineering 33