First-Order Logic

• Idea:
Foundations of Artificial Intelligence – Don’t treat propositions as “atomic” entities.
• First-Order Logic:
– Objects: cs472, fred, ph219, emptylist …
First-Order Logic – Relations/Predicates: is_Man(fred), Located(cs472, ph219) …
• Note: Relations typically correspond to verbs
CS472 – Fall 2007 – Functions: Pair(search,Pair(learning,Pair(kbsystems, emptylist)))
Thorsten Joachims – Connectives: ∧, ∨ , ¬, ⇒, ⇔
– Quantifiers:
• Universal: ∀ x: ( is_Man(x) ⇒ is_Mortal(x) )
• Existential: ∃ y: ( is_Father(y, fred) )

Example:
Representing Facts in First-Order Logic
Example: Proof
Knowledge base:
1. Lucy* is a professor
• is-prof(lucy)
2. All professors are people. • ∀ x ( is-prof(x) → is-person(x) )
• is-dean(fuchs)
3. Fuchs is the dean.
• ∀ x (is-dean(x) Æ is-prof(x))
4. Deans are professors. • ∀ x (∀ y ( is-prof(x) ∧ is-dean(y) → is-friend-of(y,x) ∨ ¬ knows(x, y) ) )
5. All professors consider the dean a friend or don’t know him. • ∀ x (∃ y ( is-friend-of (y, x) ) )
• ∀ x (Vy (is-person(x) ∧ is-person(y) ∧ criticize (x,y) → ¬ is-friend-of (y,x)))
6. Everyone is a friend of someone. • criticize(lucy,fuchs)
7. People only criticize people that are not their friends. Question: Is Fuchs no friend of Lucy?
8. Lucy criticized Fuchs. ¬ is-friend-of(fuchs,lucy)

* Name changed for privacy reasons.

Knowledge Engineering Inference Procedures: Theoretical Results

• There exist complete and sound proof procedures for propositional
1. Identify the task. and FOL.
– Propositional logic
2. Assemble the relevant knowledge. • Use the definition of entailment directly. Proof procedure is
3. Decide on a vocabulary of predicates, functions, and exponential in n, the number of symbols.
constants. • In practice, can be much faster…
4. Encode general knowledge about the domain. • Polynomial-time inference procedure exists when KB is expressed
as Horn clauses:
5. Encode a description of the specific problem instance. where the Pi and Q are non-negated atoms.
6. Pose queries to the inference procedure and get answers. – First-Order logic
7. Debug the knowledge base. • Godel’s completeness theorem showed that a proof procedure
exists…
• But none was demonstrated until Robinson’s 1965 resolution
algorithm.
• Entailment in first-order logic is semidecidable.

1
 Algorithm: Resolution Proof
Resolution Rule of Inference
• Negate the theorem to be proved, and add the result to the
General Rule: knowledge base.
• Bring knowledge base into conjunctive normal form (CNF)
– CNF: conjunctions of disjunctions
– Each disjunction is called a clause.

Note: Eij can be negated. • Until there is no resolvable pair of clauses,

– Find resolvable clauses and resolve them.
Example: – Add the results of resolution to the knowledge base.
– If NIL (empty clause) is produced, stop and report that the
(original) theorem is true.
• Report that the (original) theorem is false.

Resolution Example: Propositional Logic Resolution Example: FOL

• To prove: ¬ P Example: Prove bird (tweety)
• Transform Knowledge Base into CNF
Axioms: Regular CNF
1:
2:
• Proof
1. ¬P∨Q Sentence 1 3:
2. ¬Q∨R Sentence 2
3. ¬R Sentence 3 4:
4. P Assume opposite Resolution Proof
5. Q Resolve 4 and 1
6. R Resolve 5 and 2 1. Resolve 3 and 1, specializing (i.e. “unifying”) tweety for x.
7. nil Resolve 6 with 3 Add ¬feathers(tweety)
2. Resolve 4 and 2. Add NIL.

Resolution Theorem Proving Unification

Properties of Resolution Theorem Proving: Unify procedure: Unify(P,Q) takes two atomic (i.e. single
predicates) sentences P and Q and returns a substitution
that makes P and Q identical.
– sound (for propositional and FOL)
Rules for substitutions:
– (refutation) complete (for propositional and FOL) – Can replace a variable by a constant.
– Can replace a variable by a variable.
Procedure may seem cumbersome but note that can be – Can replace a variable by a function expression, as long
easily automated. Just “smash” clauses until empty as the function expression does not contain the variable.
clause or no more new clauses.
Unifier: a substitution that makes two clauses resolvable.

2
 Unification - Purpose Unification (example)
Who does John hate?
Given: ∃ x: Hates (John, x)
Knowledge base (in clause form):
¬ Knows (John, x) ∨ Hates (John, x) 1. ¬ Knows (John, v) ∨ Hates (John, v)
Knows (John, Jim) 2. Knows (John, Jim)
Derive: 3. Knows (y, Leo)
4. Knows (z, Mother(z))
Hates (John, Jim) 5. ¬ Hates (John, x) (since ¬ ∃ x: Hates (John, x) Ù∀ x: ¬Hates(John,x))
Resolution with 5 and 1:
Unification: unify(Hates(John, x), Hates(John, v)) = {x/v}
6. ¬ Knows (John, v)
Resolution with 6 and 2:
Need unifier {x/Jim} for resolution to work. unify(Knows(John, v), Knows(John, Jim))= {v/Jim}
or resolution with 6 and 3:
Add to knowledge base: unify(Knows (John, v), Knows (y, Leo)) = {y/John, v/Leo}
or Resolution with 6 and 4:
unify(Knows (John, v), Knows (z, Mother(z))) = {z/John, v/Mother(z)}
Answers:
1. Hates(John,x) with {x/v, v/Jim} (i.e. John hates Jim)
2. Hates(John,x) with {x/v, y/John, v/Leo} (i.e. John hates Leo)
3. Hates(John,x) with {x/v, v/Mother(z), z/John} (i.e. John hates his mother)

Converting More Complicated Sentences to

Most General Unifier CNF
In cases where there is more than one substitution choose the one that
makes the least commitment (most general) about the bindings.
UNIFY (Knows (John,x), Knows (y,z))
= {y / John, x / z}
not {y / John, x / z, z / Freda}
not {y / John, x / John, z / John}
…
See R&N for general unification algorithm. O(n2) with Refutation

1. Eliminate Implications 2. Move negations down to the atomic formulas

Equivalence Transformations:
Substitute ¬ E1 ∨ E2 for E1 → E2

Result:

3
 3. Eliminate Existential Quantifiers: 4. Rename variables as necessary
Skolemization
We want no two variables of the same name.
Harder cases:

There is one argument for each universally quantified

variable whose scope contains the Skolem function.

Easy case:

5. Move the universal quantifiers to the left 6. Move disjunctions down to the literals

This works because each quantifier uses a unique variable name.

8. Rename all variables, as necessary, so no

7. Eliminate the conjunctions two have the same name

4
 Algorithm: Putting Axioms into Clausal Form
1. Eliminate the implications.
9. Eliminate the universal quantifiers
2. Move the negations down to the atomic formulas.
3. Eliminate the existential quantifiers.
4. Rename the variables, if necessary.
5. Move the universal quantifiers to the left.
6. Move the disjunctions down to the literals.
7. Eliminate the conjunctions.
8. Rename the variables, if necessary.
9. Eliminate the universal quantifiers.

Resolution Proofs as Search Strategies for Selecting Clauses

• Search Problem unit-preference strategy: Give preference to resolutions
– States: Content of knowledge base in CNF involving the clauses with the smallest number of literals.
– Initial state: Knowledge base with negated theorem to prove
– Successor function: Resolution inference rule with unify set-of-support strategy: Try to resolve with the negated theorem
– Goal test: Does knowledge base contain the empty clause ’nil’ or a clause generated by resolution from that clause.
• Search Algorithm
– Depth first search (used in PROLOG) subsumption: Eliminates all sentences that are subsumed (i.e.,
• Note: Possibly infinite state space more specific than) an existing sentence in the KB.
• Example:
– IsPerson(Fred)
May still require exponential time.
– IsPerson(y) Æ IsPerson(mother(y))
– Goal: ∃ x: IsPerson(x)
– Answers: {x/Fred} and {x/mother(Fred)} and
{x/mother(mother(Fred))} and …

Original Sentences (Plus Background Knowledge)

Example
Jack owns a dog.
Every dog owner is an animal lover.
No animal lover kills an animal.
Either Jack or Curiosity killed the cat, who is named Tuna.
Did Curiosity kill the cat?

5
 Proof by Resolution
Conjunctive Normal Form ¬ kills(Curiosity,Tuna) kills(Jack,Tuna) ∨ kills(Curiosity,Tuna)
{}
(D is a placeholder for the dogs unknown name
(i.e. Skolem symbol/function). Think of D like kills(Jack,Tuna) ¬ AnimalLover(w) ∨ ¬ Animal(y) ∨ ¬ kills(w,y)
“JohnDoe”)
{w/Jack, y/Tuna}

Animal(z) ∨ ¬ Cat(z) ¬ AnimalLover(Jack) ∨ ¬ Animal(Tuna)

{z/Tuna}

¬ AnimalLover(Jack) ∨ ¬ Cat(Tuna) Cat(Tuna)

{}
¬ AnimalLover(Jack) ¬ Dog(y) ∨ ¬ Owns(x,y) ∨ AnimalLover(x)
{x/Jack}
¬ Dog(D) ¬ Dog(y) ∨ ¬ Owns(Jack,y) ¬ Owns(Jack,D)
{y/D}
Owns(Jack,D)
NIL

Proofs can be Lengthy Successes in Rule-Based Reasoning

A relatively straightforward KB can quickly overwhelm general resolution Expert systems
methods.
• DENDRAL (Buchanan et al., 1969)
Resolution strategies reduce the problem somewhat, but not completely.

As a consequence, many practical Knowledge Representation formalisms • MYCIN (Feigenbaum, Buchanan, Shortliffe)
in AI use a restricted form and specialized inference.
• PROSPECTOR (Duda et al., 1979)
– Logic programming (Prolog)

– Production systems • R1 (McDermott, 1982)

– Frame systems and semantic networks

– Description logics

Successes in Rule-Based Reasoning Successes in Rule-Based Reasoning

• MYCIN (Feigenbaum, Buchanan, Shortliffe)
• DENDRAL (Buchanan et al., 1969)
– Diagnosis of blood infections
– Infers molecular structure from the information provided by
– 450 rules; performs as well as experts
a mass spectrometer
– Incorporated certainty factors
– Generate-and-test method

6
 Successes in Rule-Based Reasoning Cognitive Modeling with Rule-Based Systems

• PROSPECTOR (Duda et al., 1979) SOAR is a general architecture for building intelligent
– Correctly recommended exploratory drilling at geological site systems.
– Rule-based system founded on probability theory – Long term memory consists of rules
– Working memory describes current state
• R1 (McDermott, 1982)
– All problem solving, including deciding what rule to
– Designs configurations of computer components
execute, is state space search
– About 10,000 rules
– Successful rule sequences are chunked into new rules
– Uses meta-rules to change context
– Control strategy embodied in terms of meta-rules

Prolog (Programming in Logic) Programming in Prolog

• What is Prolog? • Path Finding
– Full-featured programming language
– Programs consist of logical formulas path(Node1,Node2) :- edge(Node1,Node2).
– Running a program means proving a theorem path(Node1,Node2) :-
• Syntax of Prolog edge(Node1,SomeNode),
– Predicates, objects, and functions:
• cat(tuna), append(a,pair(b)) path(SomeNode,Node2).
– Variables: X, Y, List (capitalized) edge(ith,lga).
– Facts:
• university(cornell). edge(ith,phl).
• prepend(a,pair(a,X)). edge(phl,sfo).
– Rules:
• animal(X) :- cat(X). edge(lga,ord).
• student(X) :- person(X), enrolled(X,Y), university(Y).
Æ implication “:-” with single predicate on left and only non-negated predicates
• Query
on the right. All variables implicitly “forall” quantified. – path(ith,ord).
– Queries:
• student(X).
– path(ith,X).
Æ All variables implicitly “exists” quantified.

Programming in Prolog Programming in Prolog

• Data structures: Lists Example: Symbolic derivatives (http://cs.wwc.edu/~cs_dept/KU/PR/Prolog.html)
length([],0).
length([H|T],N) :- length(T,M), N is M+1. % deriv(Polynomial, variable, derivative)
% dc/dx = 0
deriv(C,X,0) :- number(C).
member(X,[X|List]). % dx/dx} = 1
member(X,[Element|List]) :- member(X,List). deriv(X,X,1).
% d(cv)/dx = c(dv/dx)
append([],List,List). deriv(C*U,X,C*DU) :- number(C), deriv(U,X,DU).
append([Element|L1],L2,[Element|L1L2]) :- % d(u v)/dx = u(dv/dx) + v(du/dx)
append(L1,L2,L1L2). deriv(U*V,X,U*DV + V*DU) :- deriv(U,X,DU), deriv(V,X,DV).
• Query: % d(u ± v)/dx = du/dx ± dv/dx
– length([a,b,c],3). deriv(U+V,X,DU+DV) :- deriv(U,X,DU), deriv(V,X,DV).
– length([a,b,c],X). deriv(U-V,X,DU-DV) :- deriv(U,X,DU), deriv(V,X,DV).
– member(b,[a,b,c]). % du^n/dx = nu^{n-1}(du/dx)
deriv(U^+N,X,N*U^+N1*DU) :- N1 is N-1, deriv(U,X,DU).
– member(X,[a,b,c]).

7
 Programming in Prolog Programming in Prolog
• Towers of Hanoi: move N disks from pin a to pin b • 8-Queens:
using pin c. solve(P) :-
hanoi(N):-hanoi(N, a, b, c). perm([1,2,3,4,5,6,7,8],P),
combine([1,2,3,4,5,6,7,8],P,S,D),
hanoi(0,A,B,C). all_diff(S),
hanoi(N,FromPin,ToPin,UsingPin):- all_diff(D).
M is N-1, combine([X1|X],[Y1|Y],[S1|S],[D1|D]) :-
hanoi(M,FromPin,UsingPin,ToPin), S1 is X1 +Y1,
move(FromPin,ToPin), D1 is X1 - Y1,
hanoi(M,UsingPin,ToPin,FromPin). combine(X,Y,S,D).
move(From,To):- combine([],[],[],[]).
write([move, disk from, pin, From, to, pin,
ToPin]),nl. all_diff([X|Y]) :- \+member(X,Y), all_diff(Y).
all_diff([X]).

Properties of Knowledge-Based Systems

Advantages
1. Expressibility*
2. Simplicity of inference procedures*
3. Modifiability*
4. Explainability
5. Machine readability
6. Parallelism*
Disadvantages
1. Difficulties in expressibility
2. Undesirable interactions among rules
3. Non-transparent behavior
4. Difficult debugging
5. Slow
6. Where does the knowledge base come from???