Lecture 14: Logic II

Syntax
formula

Review
Semantics

models

Inference rules

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

Inference task
Inference task Input: Knowledge base KB (e.g., {Rain, Rain Wet}) Query formula f (e.g., Wet) Output: Whether KB entails f (KB |= f )? (e.g., yes)

Review: formulas
Propositional logic: any combination of symbols

(Rain Snow) (Trac Peaceful) Wet

Propositional logic with only Horn clauses: [premises implies conclusion]

restricted to

(Rain Snow) Trac

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

Review: inference
Inference algorithm: (repeatedly apply inference rules) KB f
Formulas allowed

Approaches
Inference rule Complete?

Propositional logic (only Horn clauses) Modus ponens yes

Denition: Modus ponens inference rule p1 , , pk , (p1 pk ) q q

Propositional logic

Modus ponens no

Propositional logic

resolution

yes

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

Roadmap
Resolution in propositional logic

Horn clauses and disjunction

With implication AC AB C With disjunction A C A B C

First-order logic

Horn clauses: at most one positive propositional symbol Clause: disjunction of literals Literal: either p or p, where p is a propositional symbol

Other logics

Modus ponens (rewritten):

A, AC C

Intuition: cancel out A and A

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

Resolution
General clauses have any number of literals: A B C D E F Denition: resolution inference rule f1 fn p, p g1 gm f1 fn g1 gm Example: Rain Snow, Snow Trac Rain Trac Snow, Snow false
CS221: Articial Intelligence (Autumn 2013) - Percy Liang 8

Soundness of resolution
Rain Snow Snow Trac (resolution rule) Rain Trac
M(Rain Snow)M(Snow Trac)?M(Rain Trac)
Snow 0 1 Rain, Trac Rain, Trac 0,0 0,1 1,0 1,1 0,0 0,1 1,0 1,1 Snow 0 1 Rain, Trac 0,0 0,1 1,0 1,1 Snow 0 1

Sound!

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

Conjunctive normal form

Denition: conjunctive normal form (CNF) A CNF formula is a conjunction of clauses. Example: (A B C ) (B D) Equivalent: knowledge base where each formula is a clause Proposition: conversion to CNF Every formula f in propositional logic can be converted into an equivalent CNF form f : M(f ) = M(f )

Conversion to CNF: example

Initial formula: (Summer Snow) Bizzare Remove implication (): (Summer Snow) Bizzare Push negation ( ) inwards (de Morgan): (Summer Snow) Bizzare Remove double negation: (Summer Snow) Bizzare Distribute over : (Summer Bizzare) (Snow Bizzare)

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

10

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

11

Conversion to CNF: general

Inference rules: Eliminate : Eliminate :
f g (f g )(g f ) f g f g (f g ) f g (f g ) f g f f

Resolution algorithm
Recall: KB operations boil down to satisability KB |= f KB {f } is unsatisable

Move inwards: Move inwards:

KB |= f

KB {f } is unsatisable

Eliminate double negation: Distribute over :

Algorithm: resolution-based inference Convert all formulas into CNF. Repeatedly apply resolution rule. Return unsatisable i derive false.

f (g h ) ( f g ) (g h )

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

12

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

13

Resolution: example
Knowledge base (is it satisable?): KB = {A (B C ), A, B, C } Convert to CNF: KB = {A B C, A, B, C } Repeatedly apply resolution rule:
p1 ,

Time complexity
Denition: Modus ponens inference rule , pk , (p1 pk ) q q

Each rule application adds clause with one propositional symbol linear time
Denition: resolution inference rule

false C BC A B C
CS221: Articial Intelligence (Autumn 2013) - Percy Liang

C B

f1 fn h, h g1 gm f1 fn g1 gm

Each rule application adds clause with many propositional symbols exponential time

A
14 CS221: Articial Intelligence (Autumn 2013) - Percy Liang 15

Unsatisable!

Summary
Horn clauses any clauses

Roadmap
Resolution in propositional logic

Modus ponens

resolution First-order logic

linear time

exponential time

less expressive

more expressive Other logics

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

16

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

17

Limitations of propositional logic

Alice and Bob both know arithmetic.
AliceKnowsArithmetic BobKnowsArithmetic

Limitations of propositional logic

All students know arithmetic.
AliceIsStudent AliceKnowsArithmetic BobIsStudent BobKnowsArithmetic ...

All students know arithmetic.

AliceIsStudent AliceKnowsArithmetic BobIsStudent BobKnowsArithmetic ...

Propositional logic is very clunky. Whats missing? Objects and relations: propositions (e.g., AliceKnowsArithmetic) have more internal structure (alice, Knows, arithmetic) Quantiers and variables: all is a quantier which applies to each person, dont want to enumerate them all...

Every even integer greater than 2 is the sum of two primes. ???

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

18

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

19

First-order logic
Syntax
formula models

First-order logic: examples

Alice and Bob both know arithmetic.

Semantics

Knows(alice, arithmetic) Knows(bob, arithmetic)

All students know arithmetic.

Inference rules

x Student(x) Knows(x, arithmetic)

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

20

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

21

Syntax of rst-order logic

Terms (refer to objects): Constant symbol (e.g., arithmetic) Variable (e.g., x) Function of terms (e.g., Sum(3, 4)) Formulas (refer to truth values): Atomic formulas (atoms): predicate applied to terms (e.g., Knows(x, arithmetic)) Connectives applied to formulas (e.g., Student(x) Knows(x, arithmetic)) Quantiers applied to formulas (e.g., x Student(x) Knows(x, arithmetic))
CS221: Articial Intelligence (Autumn 2013) - Percy Liang 22

Quantiers
Universal quantication (): Every student knows arithmetic. x Student(x)Knows(x, arithmetic) Existential quantication (): Some student knows arithmetic. x Student(x)Knows(x, arithmetic) Note the dierent connectives!

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

23

Quantiers
Universal quantication (): Think conjunction: x P (x) is like P (A) P (B ) Existential quantication (): Think disjunction: x P (x) is like P (A) P (B ) Some properties: x P (x) equivalent to x P (x) x y Knows(x, y ) dierent from y x Knows(x, y )

Some examples of rst-order logic

There is some course that every student has taken.
y Course(y ) [x Student(x) Takes(x, y )]

Every even integer greater than 2 is the sum of two primes.

x EvenInt(x) Greater(x, 2) y z Equals(x, Sum(y, z )) Prime(y ) Prime(z )

If a student takes a course and the course covers some concept, then the student knows that concept.
x y z (Student(x) Takes(x, y ) Course(y ) Covers(y, z )) Knows(x, z )

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

24

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

25

First-order logic
Syntax
formula models

Models in rst-order logic

Recall a model represents a possible situation in the world. Propositional logic: Model w maps propositional symbols to truth values.
w = {AliceKnowsArithmetic : 1, BobKnowsArithmetic : 0}

Semantics

First-order logic: Model w maps atomic formulas to truth values (this is almost right, but not quite).
w = {Knows(alice, arithmetic) : 1, Knows(bob, arithmetic) : 0}

Inference rules

Problem is that if bob and robert refer to the same person, then w(Knows(bob, arithmetic)) must equal w(Knows(robert, arithmetic))

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

26

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

27

Graph representation of a model

A model w as be represented a directed graph (if only have binary predicates): w bob alice o1
Knows

Models in rst-order logic

Denition: model in rst-order logic A model w in rst-order logic maps: constant symbols to objects
w(alice) = o1 , w(bob) = o2 , w(arithmetic) = o3

o2 robert
Knows

predicate symbols to tuples of objects

w(Knows) = {(o1 , o3 ), (o2 , o3 ), . . . }

o3 arithmetic Nodes are objects, labeled with constant symbols Directed edges are relations, labeled with predicate symCS221: Articial Intelligence (Autumn 2013) - Percy Liang bols

28

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

29

A restriction on models
John and Bob are students. Student(john) Student(bob)
w1
Student Student

Propositionalization
If one-to-one mapping between constant symbols and objects (unique names and domain closure), rst-order logic is syntactic sugar for propositional logic:

w2
Student

w3
Student Student

Knowledge base in rst-order logic Student(alice) Student(bob) x Student(x) Person(x) x Student(x) Creative(x)
Knowledge base in propositional logic Student(alice) Student(bob) (Student(alice) Person(alice)) (Student(bob) Person(bob)) (Student(alice) Creative(alice)) (Student(bob) Creative(bob))

john o1

o2 bob

john o1 bob

john o1

o2

o3 bob

Unique names assumption: Each object has at most one constant symbol. This rules out w2 . Domain closure: Each object has at least one constant symbol. This rules out w3 . Point: constant symbol
CS221: Articial Intelligence (Autumn 2013) - Percy Liang

object Point: use any inference algorithm for propositional logic!

30 CS221: Articial Intelligence (Autumn 2013) - Percy Liang 31

First-order logic
Syntax
formula models

Denite clauses
Denition: denite clause (rst-order logic) A denite clause has the following form: x1 xn (a1 ak ) b for atomic formulas a1 , . . . , ak , b and variables x1 , . . . , xn . Example:

Semantics

Inference rules

x y z (Takes(x, y ) Covers(y, z )) Knows(x, z ) (if propositionalize, get one formula for each x, y, z )

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

32

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

33

Substitution and unication

Goal: dene inference rules that work on formulas with quantiers Example: Given P (alice) and x P (x) Q(x). Infer Q(alice)? Problem: P (x) and P (alice) dont match exactly. Two concepts:

Substitution
Denition: Substitution A substitution is a mapping from variables to constant symbols or other variables. Subst[, f ] returns the result of performing substitution on f . Examples:
Subst[{x/alice}, P (x)] = P (alice)

Substitution: morph a formula into another Unication: make two formulas the same
Subst[{x/alice, y/z }, P (x) K (x, y )] = P (alice) K (alice, z )

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

34

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

35

Unication
Denition: Unication Unication takes two formulas f and g and returns a substitution which is the most general unier: Unify[f, g ] = such that Subst[, f ] = Subst[, g ] or fail if no exists. Examples:
Unify[Knows(alice, arithmetic), Knows(x, arithmetic)] = {x/alice} Unify[Knows(alice, y ), Knows(x, z )] = {x/alice, y/z } Unify[Knows(alice, y ), Knows(bob, z )] = fail Unify[Knows(alice, y ), Knows(x, F (x))] = {x/alice, y/F (alice)}

Modus ponens
Denition: Modus ponens (rst-order logic) a1 , ak x1 xn (a1 ak ) b , b

Get most general unier on premises: = Unify[a1 ak , a1 ak ] Apply to conclusion: Subst[, b] = b

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

36

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

37

Modus ponens example

Example: Modus ponens in rst-order logic Premises:
Takes(alice, cs221) Covers(cs221, mdp) x y z Takes(x, y ) Covers(y, z ) Knows(x, z )

Complexity
x y z P (x, y, z ) Each application of Modus ponens produces an atomic formula. If no function symbols, number of atomic formulas is at most (num-constant-symbols)(maximum-predicate-arity) If there are function symbols (e.g., F ), then innite...
Q(a) Q(F (a)) Q(F (F (a))) Q(F (F (F (a))))

Conclusion:
= {x/alice, y/cs221, z/mdp} Derive Knows(alice, mdp)

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

38

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

39

Complexity
Theorem: completeness Modus ponens is complete for rst-order logic with only Horn clauses. Theorem: semi-decidability First-order logic (even restricted to only Horn clauses) is semi-decidable. If KB |= f , forward inference on complete inference rules will prove f in nite time. If KB |= f , no algorithm can show this in nite time.

Resolution
Recall: First-order logic includes non-Horn clauses

x Student(x) y Knows(x, y )

High-level strategy (same as in propositional logic): Convert all formulas to CNF Repeatedly apply resolution rule

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

40

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

41

Conversion to CNF
Input: x (y Animal(y ) Loves(x, y )) y Loves(y, x) Output:
(Animal(Y (x)) Loves(Z (z ), x)) (Loves(x, Y (x)) Loves(Z (z ), x))

Conversion to CNF
Anyone who likes all animals is liked by someone. Input: x (y Animal(y ) Loves(x, y )) y Loves(y, x) Eliminate implications (old): x (y Animal(y ) Loves(x, y )) y Loves(y, x) Push inwards, eliminate double negation (old): x (y Animal(y ) Loves(x, y )) y Loves(y, x) Standardize variables (new): x (y Animal(y ) Loves(x, y )) z Loves(z, x)

New to rst-order logic: All variables (e.g., x) have universal quantiers by default Introduce Skolem functions (e.g., Y (x)) to represent existential quantied variables

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

42

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

43

Conversion to CNF (continued)

x (y Animal(y ) Loves(x, y )) z Loves(z, x) Replace existentially quantied variables with Skolem functions (new):
x [Animal(Y (x)) Loves(x, Y (x))] Loves(Z (x), x)

Resolution
Denition: Resolution rule (rst-order logic) f1 fn h1 , h2 g1 gm Subst[, f1 fn g1 gm ] where = Unify[h1 , h2 ].

Distribute over (old):

x [Animal(Y (x)) Loves(Z (x), x)] [Loves(x, Y (x)) Loves(Z (x), x)]

Example: resolution
Animal(Y (x)) Loves(Z (x), x), Loves(u, v ) Kills(u, v ) Animal(Y (x)) Kills(Z (x), x)

Remove universal quantiers (new):

[Animal(Y (x)) Loves(Z (x), x)] [Loves(x, Y (x)) Loves(Z (x), x)]

Interpretation: Y (x) represents animal that x doesnt like, Z (x) represents person who likes x

Substitution: = {u/Z (x), v/x}.

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

44

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

45

Summary
Propositional logic model checking First-order logic n/a propositionalization Modus ponens (Horn clauses) resolution (general) Modus ponens++ (Horn clauses) resolution++ (general) ++: unication and substitution Key idea: variables in rst-order logic Variables yield compact knowledge representations.
CS221: Articial Intelligence (Autumn 2013) - Percy Liang 46 CS221: Articial Intelligence (Autumn 2013) - Percy Liang

Roadmap
Resolution in propositional logic

First-order logic

Other logics

47

Motivation
Goal: represent knowledge and perform inferences Why use anything besides propositional or rst-order logic? Expressiveness: Temporal logic: express time Modal logic: express beliefs Higher-order logic: generalized quantiers Notational convenience, computational eciency: Description logic

Temporal logic
Barack Obama is the US president. President(BarackObama, US) George Washington was the US president. P President(GeorgeWashington, US) Some woman will be the US president. F x Female(x) President(x, US)

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

48

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

49

Temporal logic
Point: all formulas interpreted at a current time. The following operators change the current time and quantify over it (think of Pf as t [t < now] f (t)): P f : f held at some point in the past F f : f will hold at some point in the future H f : f held at every point in the past G f : f will hold at every point in the future Every student will at some point never be a student again. x.Student(x) FGStudent(x)

Modal logic for propositional attitudes

Alice believes one plus one is two. Believes(alice, Equals(Sum(1, 1), 2))?? Alice believes Boston is a city. Believes(alice, City(boston))?? Problem: Equals(Sum(1, 1), 2) is true, City(boston) is true, but two are not interchangeable in this context.

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

50

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

51

Modal logic for propositional attitudes

Solution: interpret every formula with respect to a possible world, operator Balice f interprets f according to Alices world Simple:

Higher-order logic: lambda calculus

Alice has visited some museum. x Museum(x) Visited(alice, x)

Balice City(boston)

More complex: Alice has visited at least 10 museums. x Museum(x)Visited(alice, x): boolean function representing set of museums Alice has visited Count(x Museum(x)Visited(alice, x)) 10 Higher-order logic allows us to model these generalized quantiers.

LivesIn(alice, boston) Balice City(boston)

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

52

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

53

Description logic
People with at least three sons who are all unemployed and married to doctors, and at most two daughters who are professors...are weird.

Limitations
In logic, a formula carves out subset of models. x y z Takes(x, y ) Covers(y, z ) Knows(x, z )

First-order logic + lambda calculus:

x Person(x) Count(y Son(x, y ) Unemployed(y ) z Spouse(y, z ) Doctor(z )) 3 Count(y Daughter(x, y ) Professor(y )) 2 Weird(x)

Description logic:
Person ( 3 Son.Unemployed Spouse.Doctor) Weird

In reality, there is uncertainty. Key idea: distribution Model a distribution over models P(W = w; ).

( 2 Daughter.Professor)

Advantages: more compact, decidable

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

54

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

55

Markov logic
Example: Markov logic network (MLN) formula G(a) G(b) x G(x) H (x) weight 0.2 5.1 1.2 Propositionalize:

Markov logic
Assume unique names + domain closure.

Example: grounded Markov logic network formula G(a) G(b) G(a) H (a) G(b) H (b) G(c) H (c) weight 0.2 5.1 1.2 1.2 1.2
57

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

56

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

Markov logic
Recall propositional logic encodes a factor graph: e1.2[G(a)H (a)] e0.2[G(a)=1] G(a) H (b)

Summary
Logic is used to represent knowledge and perform inferences on it Considerations: expressiveness, notational convenience, inferential complexity Propositional logic with denite clauses, propostional logic, description logic, rst-order logic, temporal logic, modal logic, higher-order logic Markov logic: marry logic (abstract reasoning by working on formulas) and probability (maintaining uncertainty)

e1.2[G(b)H (b)] e
5.1[G(b)=1]

G(b)

H (b)

e1.2[G(c)H (c)] G(c) H (c)

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

58

CS221: Articial Intelligence (Autumn 2013) - Percy Liang

59