Induction as Inverted Deduction

Lecture Outline:

• Induction as Inverted Deduction

• Propositional Resolution

• Inverted Propositional Resolution

• First Order Resolution

• Inverted First Order Resolution

• Progol: An ILP Language

Reading:
Chapter 10.6-10.7 of Mitchell
Inductive Logic Programming: Theory and Methods, S. Muggleton and L. de Raedt (on
course web page)
CProgol4.4: a tutorial introduction, S. Muggleton and J. Firth (on course web page)

COM3250 / 6170 1 2010-2011

 Induction as Inverted Deduction

• A second approach to inductive logic programming is based on the observation that induction
is the inverse of deduction.
Induction is, in fact, the inverse operation of deduction, and cannot be conceived to
exist without the corresponding operation, so that the question of relative importance
cannot arise. Who thinks of asking whether addition or subtraction is the more
important process in arithmetic? But at the same time much difference in difficulty may
exist between a direct and inverse operation; . . . it must be allowed that inductive
investigations are of a far higher degree of difficulty and complexity than any questions
of deduction. . . . (Jevons 1874)

COM3250 / 6170 2 2010-2011

 Induction as Inverted Deduction

• A second approach to inductive logic programming is based on the observation that induction
is the inverse of deduction.
Induction is, in fact, the inverse operation of deduction, and cannot be conceived to
exist without the corresponding operation, so that the question of relative importance
cannot arise. Who thinks of asking whether addition or subtraction is the more
important process in arithmetic? But at the same time much difference in difficulty may
exist between a direct and inverse operation; . . . it must be allowed that inductive
investigations are of a far higher degree of difficulty and complexity than any questions
of deduction. . . . (Jevons 1874)
• More formally, induction is finding a hypothesis h such that

(∀hxi , f (xi )i ∈ D) B ∧ h ∧ xi ⊢ f (xi )

where
– D is training data
– xi is ith training instance
– f (xi ) is the target function value for xi
– B is other background knowledge

COM3250 / 6170 2-a 2010-2011

 Induction as Inverted Deduction

• A second approach to inductive logic programming is based on the observation that induction
is the inverse of deduction.
Induction is, in fact, the inverse operation of deduction, and cannot be conceived to
exist without the corresponding operation, so that the question of relative importance
cannot arise. Who thinks of asking whether addition or subtraction is the more
important process in arithmetic? But at the same time much difference in difficulty may
exist between a direct and inverse operation; . . . it must be allowed that inductive
investigations are of a far higher degree of difficulty and complexity than any questions
of deduction. . . . (Jevons 1874)
• More formally, induction is finding a hypothesis h such that

(∀hxi , f (xi )i ∈ D) B ∧ h ∧ xi ⊢ f (xi )

where
– D is training data
“For each training data instance, the instance’s
– xi is ith training instance target classification is logically entailed by the
– f (xi ) is the target function value for xi background knowledge, together with hypoth-
– B is other background knowledge esis and the instance itself”

COM3250 / 6170 2-b 2010-2011

 Induction as Inverted Deduction

• A second approach to inductive logic programming is based on the observation that induction
is the inverse of deduction.
Induction is, in fact, the inverse operation of deduction, and cannot be conceived to
exist without the corresponding operation, so that the question of relative importance
cannot arise. Who thinks of asking whether addition or subtraction is the more
important process in arithmetic? But at the same time much difference in difficulty may
exist between a direct and inverse operation; . . . it must be allowed that inductive
investigations are of a far higher degree of difficulty and complexity than any questions
of deduction. . . . (Jevons 1874)
• More formally, induction is finding a hypothesis h such that

(∀hxi , f (xi )i ∈ D) B ∧ h ∧ xi ⊢ f (xi )

where
– D is training data
“For each training data instance, the instance’s
– xi is ith training instance target classification is logically entailed by the
– f (xi ) is the target function value for xi background knowledge, together with hypoth-
– B is other background knowledge esis and the instance itself”
• Suggests we design an inductive algorithm by inverting operators for automated deduction . . .

COM3250 / 6170 2-c 2010-2011

 Induction as Inverted Deduction: Example

• Concept to be learned: “pairs of people, hu, vi such that child of u is v”

COM3250 / 6170 3 2010-2011

 Induction as Inverted Deduction: Example

• Concept to be learned: “pairs of people, hu, vi such that child of u is v”

• Suppose we are given
– one training example
∗ describing two people Tim and Sharon in terms of their gender and their relation in
terms of the predicate f ather
∗ giving the target predicate Child(Tim, Sharon) for these two people
– background knowledge asserting that “if u is the father of v then u is a parent of v”

COM3250 / 6170 3-a 2010-2011

 Induction as Inverted Deduction: Example

• Concept to be learned: “pairs of people, hu, vi such that child of u is v”

• Suppose we are given
– one training example
∗ describing two people Tim and Sharon in terms of their gender and their relation in
terms of the predicate f ather
∗ giving the target predicate Child(Tim, Sharon) for these two people
– background knowledge asserting that “if u is the father of v then u is a parent of v”

f (xi ) : Child(Tim, Sharon)

xi : Male(Tim), Female(Sharon), Father(Sharon, Tim)
B: Parent(u, v) ← Father(u, v)

COM3250 / 6170 3-b 2010-2011

 Induction as Inverted Deduction: Example

• Concept to be learned: “pairs of people, hu, vi such that child of u is v”

• Suppose we are given
– one training example
∗ describing two people Tim and Sharon in terms of their gender and their relation in
terms of the predicate f ather
∗ giving the target predicate Child(Tim, Sharon) for these two people
– background knowledge asserting that “if u is the father of v then u is a parent of v”

f (xi ) : Child(Tim, Sharon)

xi : Male(Tim), Female(Sharon), Father(Sharon, Tim)
B: Parent(u, v) ← Father(u, v)

• What h satisfies (∀hxi , f (xi )i ∈ D) B ∧ h ∧ xi ⊢ f (xi )?

COM3250 / 6170 3-c 2010-2011

 Induction as Inverted Deduction: Example

• Concept to be learned: “pairs of people, hu, vi such that child of u is v”

• Suppose we are given
– one training example
∗ describing two people Tim and Sharon in terms of their gender and their relation in
terms of the predicate f ather
∗ giving the target predicate Child(Tim, Sharon) for these two people
– background knowledge asserting that “if u is the father of v then u is a parent of v”

f (xi ) : Child(Tim, Sharon)

xi : Male(Tim), Female(Sharon), Father(Sharon, Tim)
B: Parent(u, v) ← Father(u, v)

• What h satisfies (∀hxi , f (xi )i ∈ D) B ∧ h ∧ xi ⊢ f (xi )?

h1 : Child(u, v) ← Father(v, u)
h2 : Child(u, v) ← Parent(v, u)

h1 ∧ Father(Sharon, Tim) ⊢ Child(Tim, Sharon)

Parent(u, v) ← Father(u, v) ∧ h2 ∧ Father(Sharon, Tim) ⊢ Child(Tim, Sharon)

COM3250 / 6170 3-d 2010-2011

 Induction as Inverted Deduction: Pluses + Minuses

• There are pluses and minuses to casting the learning task as a search for hypotheses h such that

(∀hxi , f (xi )i ∈ D) (B ∧ h ∧ xi ) ⊢ f (xi )

COM3250 / 6170 4 2010-2011

 Induction as Inverted Deduction: Pluses + Minuses

• There are pluses and minuses to casting the learning task as a search for hypotheses h such that

(∀hxi , f (xi )i ∈ D) (B ∧ h ∧ xi ) ⊢ f (xi )

• Positives:
– Subsumes earlier idea of finding h that “fits” training data
– Domain theory B helps define meaning of “fit” the data

B ∧ h ∧ xi ⊢ f (xi )

– Suggests algorithms that search H guided by B

COM3250 / 6170 4-a 2010-2011

 Induction as Inverted Deduction: Pluses + Minuses

• There are pluses and minuses to casting the learning task as a search for hypotheses h such that

(∀hxi , f (xi )i ∈ D) (B ∧ h ∧ xi ) ⊢ f (xi )

• Positives:
– Subsumes earlier idea of finding h that “fits” training data
– Domain theory B helps define meaning of “fit” the data

B ∧ h ∧ xi ⊢ f (xi )

– Suggests algorithms that search H guided by B

• Negatives:
– Doesn’t naturally allow for noisy data – noise can result in inconsistent constraints in h and
most logical frameworks break down when given inconsistent sets of assertions
– First order logic gives a huge hypothesis space H
∗ overfitting...
∗ intractability of calculating all acceptable h’s
– While using background knowledge B should help constrain hypothesis search, for many
ILP systems hypothesis space search increases as B is increased

COM3250 / 6170 4-b 2010-2011

 Propositional Resolution

• We are seeking mechanical inductive operators O such that

O(B, D) = h where (∀hxi , f (xi )i ∈ D) (B ∧ h ∧ xi ) ⊢ f (xi )

COM3250 / 6170 5 2010-2011

 Propositional Resolution

• We are seeking mechanical inductive operators O such that

O(B, D) = h where (∀hxi , f (xi )i ∈ D) (B ∧ h ∧ xi ) ⊢ f (xi )

• We already have mechanical deductive operators

F(A, B) = C, where A ∧ B ⊢ C

Can we invert these?

COM3250 / 6170 5-a 2010-2011

 Propositional Resolution

• We are seeking mechanical inductive operators O such that

O(B, D) = h where (∀hxi , f (xi )i ∈ D) (B ∧ h ∧ xi ) ⊢ f (xi )

• We already have mechanical deductive operators

F(A, B) = C, where A ∧ B ⊢ C

Can we invert these?

• The best known mechanical deductive operator is resolution
C1 : P ∨ L
C2 : ¬L ∨ R
Resolvent: P ∨ R

COM3250 / 6170 5-b 2010-2011

 Propositional Resolution

• We are seeking mechanical inductive operators O such that

O(B, D) = h where (∀hxi , f (xi )i ∈ D) (B ∧ h ∧ xi ) ⊢ f (xi )

• We already have mechanical deductive operators

F(A, B) = C, where A ∧ B ⊢ C

Can we invert these?

• The best known mechanical deductive operator is resolution
C1 : P ∨ L
C2 : ¬L ∨ R
Resolvent: P ∨ R
Treating clauses as sets of literals (i.e. implicit disjunction) resolution is defined as follows:
1. Given initial clauses C1 and C2 , find a literal L from clause C1 such that ¬L occurs in C2
2. Form the resolvent C by including all literals from C1 and C2 , except for L and ¬L. More
precisely, the set of literals occurring in the conclusion C is

C = (C1 − {L}) ∪ (C2 − {¬L})

where ∪ denotes set union, and “−” denotes set difference.

COM3250 / 6170 5-c 2010-2011
 Inverted Propositional Resolution
• Consider:
C2 Study → KnowMaterial ¬Study ∨ KnowMaterial
C1 KnowMaterial → PassExam ≡ ¬KnowMaterial ∨ PassExam
C Study → PassExam ¬Study ∨ PassExam
C : KnowMaterial V Study C : KnowMaterial V Study
2 2

C : PassExam V KnowMaterial C : PassExam V KnowMaterial

1 1

C: PassExam V Study
C: PassExam V Study

COM3250 / 6170 6 2010-2011

 Inverted Propositional Resolution
• Consider:
C2 Study → KnowMaterial ¬Study ∨ KnowMaterial
C1 KnowMaterial → PassExam ≡ ¬KnowMaterial ∨ PassExam
C Study → PassExam ¬Study ∨ PassExam
C : KnowMaterial V Study C : KnowMaterial V Study
2 2

C : PassExam V KnowMaterial C : PassExam V KnowMaterial

1 1

C: PassExam V Study
C: PassExam V Study

• So, to invert we need to go from C and C1 to C2 . A general operation for doing this is (inverted
resolution):
1. Given initial clauses C1 and C, find a literal L that occurs in clause C1 , but not in clause C.
2. Form the second clause C2 by including the following literals

C2 = (C − (C1 − {L})) ∪ {¬L}

COM3250 / 6170 6-a 2010-2011

 First Order Resolution

• Like propositional resolution, first order resolution takes two clauses C1 and C2 as input and
yields a third C as output.

Unlike propositional resolution the C1 and C2 must be related not by sharing a literal and its
negation, but by sharing a literal and negated literal that can be matched by a unifying
substitution.

COM3250 / 6170 7 2010-2011

 First Order Resolution

• Like propositional resolution, first order resolution takes two clauses C1 and C2 as input and
yields a third C as output.

Unlike propositional resolution the C1 and C2 must be related not by sharing a literal and its
negation, but by sharing a literal and negated literal that can be matched by a unifying
substitution.

• Formally, first order resolution is defined as follows:

1. Find a literal L1 from clause C1 , literal L2 from clause C2 , and substitution θ such that
L1 θ = ¬L2 θ
2. Form the resolvent C by including all literals from C1 θ and C2 θ, except for L1 θ and ¬L2 θ.
More precisely, the set of literals occurring in the conclusion C is

C = (C1 − {L1 })θ ∪ (C2 − {L2 })θ

COM3250 / 6170 7-a 2010-2011

 First Order Resolution – Example

• Example:

C1 Swan(X) → W hite(X) ¬Swan(X) ∨W hite(X)

C2 Swan( f red) ≡ Swan( f red)
C W hite( f red) W hite( f red)

Setting θ = {X/ f red} we have

C = (C1 − {L1 })θ ∪ (C2 − {L2 })θ

= ({¬Swan(X),W hite(X)} − {¬Swan(X)}){X/ f red} ∪
({Swan( f red)} − {Swan( f red)}){X/ f red}
= W hite( f red)

COM3250 / 6170 8 2010-2011

 Inverted First Order Resolution
• Can derive an inverse first order resolution operator by algebraic manipulation of the equation
expressing the definition of the first order resolvent:

C = (C1 − {L1 })θ ∪ (C2 − {L2 })θ

COM3250 / 6170 9 2010-2011

 Inverted First Order Resolution
• Can derive an inverse first order resolution operator by algebraic manipulation of the equation
expressing the definition of the first order resolvent:

C = (C1 − {L1 })θ ∪ (C2 − {L2 })θ

• First note the substitution θ can be factored into two substitutions θ1 and θ2 such that:
– θ = θ1 θ2
– θ1 contains all and only variable bindings involving variables in C1
– θ2 contains all and only variable bindings involving variables in C2
Since C1 and C2 are universally quantified they can be rewritten, if necessary, to contain no
variables in common. Hence the above definition can be rewritten as:

C = (C1 − {L1 })θ1 ∪ (C2 − {L2 })θ2

COM3250 / 6170 9-a 2010-2011

 Inverted First Order Resolution
• Can derive an inverse first order resolution operator by algebraic manipulation of the equation
expressing the definition of the first order resolvent:

C = (C1 − {L1 })θ ∪ (C2 − {L2 })θ

• First note the substitution θ can be factored into two substitutions θ1 and θ2 such that:
– θ = θ1 θ2
– θ1 contains all and only variable bindings involving variables in C1
– θ2 contains all and only variable bindings involving variables in C2
Since C1 and C2 are universally quantified they can be rewritten, if necessary, to contain no
variables in common. Hence the above definition can be rewritten as:

C = (C1 − {L1 })θ1 ∪ (C2 − {L2 })θ2

• If we restrict inverse resolution to infer clauses C2 that contain no literals in common with C1
(i.e. we prefer the shortest C2 ’s) then re-write above as:

C − (C1 − {L1 })θ1 = (C2 − {L2 })θ2

COM3250 / 6170 9-b 2010-2011

 Inverted First Order Resolution
• Can derive an inverse first order resolution operator by algebraic manipulation of the equation
expressing the definition of the first order resolvent:

C = (C1 − {L1 })θ ∪ (C2 − {L2 })θ

• First note the substitution θ can be factored into two substitutions θ1 and θ2 such that:
– θ = θ1 θ2
– θ1 contains all and only variable bindings involving variables in C1
– θ2 contains all and only variable bindings involving variables in C2
Since C1 and C2 are universally quantified they can be rewritten, if necessary, to contain no
variables in common. Hence the above definition can be rewritten as:

C = (C1 − {L1 })θ1 ∪ (C2 − {L2 })θ2

• If we restrict inverse resolution to infer clauses C2 that contain no literals in common with C1
(i.e. we prefer the shortest C2 ’s) then re-write above as:

C − (C1 − {L1 })θ1 = (C2 − {L2 })θ2

• Finally, noting L2 = ¬L1 θ1 θ−1

2 (by definition of the resolution rule) we get:

2 ∪ {¬L1 θ1 θ2 }
Inverse Resolution: C2 = (C − (C1 − {L1 })θ1 )θ−1 −1

COM3250 / 6170 9-c 2010-2011

 Inverted First Order Resolution: Example

Father (Tom, Bob) GrandChild ( y,x ) V Father ( x,z ) V Father ( z,y )

{Bob/y, Tom/z}

Father (Shannon, Tom ) GrandChild ( Bob,x) V Father ( x,Tom )

{Shannon/x}

GrandChild ( Bob, Shannon)

• Suppose we wish to learn rules for target predicate GrandChild(y, x) given

– training data D = GrandChild(Bob, Shannon)
– background information B = {Father(Shannon, Tom), Father(Tom.Bob)}

COM3250 / 6170 10 2010-2011

 Inverted First Order Resolution: Example (cont)

Father (Tom, Bob) GrandChild ( y,x ) V Father ( x,z ) V Father ( z,y )

{Bob/y, Tom/z}

Father (Shannon, Tom ) GrandChild ( Bob,x) V Father ( x,Tom )

{Shannon/x}

GrandChild ( Bob, Shannon)

• Proceed as follows:

COM3250 / 6170 11 2010-2011

 Inverted First Order Resolution: Example (cont)

Father (Tom, Bob) GrandChild ( y,x ) V Father ( x,z ) V Father ( z,y )

{Bob/y, Tom/z}

Father (Shannon, Tom ) GrandChild ( Bob,x) V Father ( x,Tom )

{Shannon/x}

GrandChild ( Bob, Shannon)

• Proceed as follows:
1. Set C = GrandChild(Bob, Shannon)

COM3250 / 6170 11-a 2010-2011

 Inverted First Order Resolution: Example (cont)

Father (Tom, Bob) GrandChild ( y,x ) V Father ( x,z ) V Father ( z,y )

{Bob/y, Tom/z}

Father (Shannon, Tom ) GrandChild ( Bob,x) V Father ( x,Tom )

{Shannon/x}

GrandChild ( Bob, Shannon)

• Proceed as follows:
1. Set C = GrandChild(Bob, Shannon)
2. Select C1 = Father(Shannon, Tom) from B

COM3250 / 6170 11-b 2010-2011

 Inverted First Order Resolution: Example (cont)

Father (Tom, Bob) GrandChild ( y,x ) V Father ( x,z ) V Father ( z,y )

{Bob/y, Tom/z}

Father (Shannon, Tom ) GrandChild ( Bob,x) V Father ( x,Tom )

{Shannon/x}

GrandChild ( Bob, Shannon)

• Proceed as follows:
1. Set C = GrandChild(Bob, Shannon)
2. Select C1 = Father(Shannon, Tom) from B
3. For inverse resolution L1 must be Father(Shannon, Tom). Selecting θ−1
2 = {Shannon/x},
C2 = (C − (C1 − {L1 })θ1 )θ−1
2 ∪ {¬L 1 θ 1 θ −1
2 }
= {GrandChild(Bob, x)} ∪ {¬Father(x, Tom)}

COM3250 / 6170 11-c 2010-2011

 Inverted First Order Resolution: Example (cont)

Father (Tom, Bob) GrandChild ( y,x ) V Father ( x,z ) V Father ( z,y )

{Bob/y, Tom/z}

Father (Shannon, Tom ) GrandChild ( Bob,x) V Father ( x,Tom )

{Shannon/x}

GrandChild ( Bob, Shannon)

• Proceed as follows:
1. Set C = GrandChild(Bob, Shannon)
2. Select C1 = Father(Shannon, Tom) from B
3. For inverse resolution L1 must be Father(Shannon, Tom). Selecting θ−1
2 = {Shannon/x},
C2 = (C − (C1 − {L1 })θ1 )θ−1
2 ∪ {¬L 1 θ 1 θ −1
2 }
= {GrandChild(Bob, x)} ∪ {¬Father(x, Tom)}
4. Repeating, with appropriate choices for L1 and θ−1
2 yields:
GrandChild(y, x) ← Father(x, z) ∧ Father(z, y)

COM3250 / 6170 11-d 2010-2011

 Inverse Resolution: Observations

• The inverse resolution operation is non-deterministic

In general, for a given target predicate C
– there are many ways to pick C1 and L1
– many ways to pick the unifying substitutions θ1 and θ2

COM3250 / 6170 12 2010-2011

 Inverse Resolution: Observations

• The inverse resolution operation is non-deterministic

In general, for a given target predicate C
– there are many ways to pick C1 and L1
– many ways to pick the unifying substitutions θ1 and θ2

• Rule learning algorithms based on inverse resolution have been developed

For example CIGOL uses sequential covering to iteratively learn a set of Horn clauses that
covers positive examples
– on each iteration a training example hxi , f (xi )i not yet covered by rules is selected
– inverse resolution is used to generate a candidate hypothesis h that satisfies

B ∧ h ∧ xi ⊢ f (xi )

where B is background knowledge plus clauses learned already

– note that this is an example-driven search, though if multiple hypotheses cover the
example, then the one with highest accuracy over further examples can be preferred
– this contrasts with FOIL which uses a generate-then-test approach

COM3250 / 6170 12-a 2010-2011

 Inverse Resolution: Observations

• The inverse resolution operation is non-deterministic

In general, for a given target predicate C
– there are many ways to pick C1 and L1
– many ways to pick the unifying substitutions θ1 and θ2

• Rule learning algorithms based on inverse resolution have been developed

For example CIGOL uses sequential covering to iteratively learn a set of Horn clauses that
covers positive examples
– on each iteration a training example hxi , f (xi )i not yet covered by rules is selected
– inverse resolution is used to generate a candidate hypothesis h that satisfies

B ∧ h ∧ xi ⊢ f (xi )

where B is background knowledge plus clauses learned already

– note that this is an example-driven search, though if multiple hypotheses cover the
example, then the one with highest accuracy over further examples can be preferred
– this contrasts with FOIL which uses a generate-then-test approach

• Another approach to using inverse resolution is P ROGOL . . .

COM3250 / 6170 12-b 2010-2011

 Progol Overview

• Inverse resolution is one way to invert deduction to derive inductive generalisations.

But, can easily lead to combinatorial explosion of candidate hypotheses due to multiple
choices for:
– input clauses/literals for inverse resolution
– unifying substitutions for inverse resolution

COM3250 / 6170 13 2010-2011

 Progol Overview

• Inverse resolution is one way to invert deduction to derive inductive generalisations.

But, can easily lead to combinatorial explosion of candidate hypotheses due to multiple
choices for:
– input clauses/literals for inverse resolution
– unifying substitutions for inverse resolution

• P ROGOL reduces this combinatorial explosion by using an alternative approach, called mode
directed inverse entailment (MDIE)
– Use inverse entailment to generate most specific h that together with background
information entails observed data
– Then perform general-to-specific search through a hypothesis space H bounded by the
most specific hypothesis and constrained by user-specified predicates

COM3250 / 6170 13-a 2010-2011

 Progol Overview

• Inverse resolution is one way to invert deduction to derive inductive generalisations.

But, can easily lead to combinatorial explosion of candidate hypotheses due to multiple
choices for:
– input clauses/literals for inverse resolution
– unifying substitutions for inverse resolution

• P ROGOL reduces this combinatorial explosion by using an alternative approach, called mode
directed inverse entailment (MDIE)
– Use inverse entailment to generate most specific h that together with background
information entails observed data
– Then perform general-to-specific search through a hypothesis space H bounded by the
most specific hypothesis and constrained by user-specified predicates

COM3250 / 6170 13-b 2010-2011

 Progol Overview (cont)

• More fully:
1. User specifies H by stating predicates, functions, and forms of arguments allowed for each
2. P ROGOL uses sequential covering algorithm:
For each hxi , f (xi )i not covered by learned rules
– Find most specific hypothesis hi s.t. B ∧ hi ∧ xi ⊢ f (xi )
∗ actually, considers only k-step entailment
– Conduct general-to-specific search of H bounded by specific hypothesis hi , choosing
hypothesis with minimum description length
– remove positive examples covered by new hypothesis

COM3250 / 6170 14 2010-2011

 Presenting a Learning Problem to Progol

Training Examples and Background Knowledge

• positive training examples are presented to Progol as ground facts are asserted in a Prolog
program. E.g.:
aunt_of(jane,henry).
aunt_of(sally,jim).
• negative training examples are presented to Progol as are positive examples, but preceded
with ‘:-’ E.g.:
:- aunt_of(henry,jane).
:- aunt_of(sally,jane).
• background knowledge, either facts or rules, is presented as in Prolog program. E.g.:
parent_of(Parent,Child) :- father_of(Parent,Child).
parent_of(Parent,Child) :- mother_of(Parent,Child).
Types Describe categories of objects in the world as it is being modelled. E.g.:
person(jane).
person(henry).
...
State that objects satisfy the unary predicates asserted of them.

COM3250 / 6170 15 2010-2011

 Presenting a Learning Problem to Progol (cont)

Modes Describe relations (predicates) between objects of given types which can be used in
• the head of hypothesized clauses (modeh declarations)
• the body of hypothesized clauses (modeb declarations)
E.g. suppose we are learning a rule for the aunt of predicate.
modeh declarations A head mode declaration might be:
:- modeh(1,aunt_of(+person,+person))?
This declaration states that:
• the head literal has predicate aunt of
• the head literal’s arguments are of type person
• arguments are input variables – i.e. these variables should be bound when clause is
called (indicated by ‘+’; ‘-’ indicates output variable; ‘#’ indicates constant required)
• the recall is 1 – the recall bounds the number of alternative solutions when the literal is
instantiated (aunt of returns a unique answer – yes or no – when its arguments are
instantiated) – can be any number or ‘*’

COM3250 / 6170 16 2010-2011

 Presenting a Learning Problem to Progol (cont)

modeb declarations Body mode declarations might be:

:- modeb(*,parent_of(-person,+person))?
:- modeb(*,parent_of(+person,-person))?
:- modeb(*,sister_of(+person,-person))?
These declarations allow the hypothesis body to contain
• parent of predicates to find ≥ 1 parents of a child
• parent of predicates to find ≥ 1 children of a parent
• sister of predicates to find ≥ 1 sisters of a person

COM3250 / 6170 17 2010-2011

 Presenting a Learning Problem to Progol (cont)

Settings Describe runtime parameters to control Progol

E.g.
:- set(posonly)?
This informs Progol to use its ‘positive example only’ evaluation mechanism.
Other settings include settings on resource bounds such as:
h – maximum depth of any proof carried out by the Prolog interpreter inside Progol (i.e. stack
depth before backtracking is forced)
r – maximum depth of resolutions allowed in any proof carried out by the Prolog interpreter
nodes – number of search nodes explored in the hypothesis search
c – maximum number of goals in body of hypothesis
i – number of iterations in constructing most specific clause, each introducing new variables
based on instantiation of output terms as specified in modeb declarations

COM3250 / 6170 18 2010-2011

 Example – Input

% Learning aunt_of from parent_of and sister_of.

% Settings
:- set(posonly)?
% Mode declarations
:- modeh(1,aunt_of(+person,+person))?
:- modeb(*,parent_of(-person,+person))?
:- modeb(*,parent_of(+person,-person))?
:- modeb(*,sister_of(+person,-person))?
% Types
person(jane).
person(henry).
person(sally).
person(jim).
person(sam).
person(sarah).
person(judy).
% Background knowledge
parent_of(Parent,Child) :- father_of(Parent,Child).
parent_of(Parent,Child) :- mother_of(Parent,Child).
father_of(sam,henry).
mother_of(sarah,jim).
sister_of(jane,sam).
sister_of(sally,sarah).
sister_of(judy,sarah).
% Examples
aunt_of(jane,henry).
aunt_of(sally,jim).
aunt_of(judy,jim).

COM3250 / 6170 19 2010-2011

 Example – Output
CProgol Version 4.4
[Noise has been set to 100%]
[Example inflation has been set to 400%]
[The posonly flag has been turned ON]
[:- set(posonly)? - Time taken 0.00s]
[:- modeh(1,aunt_of(+person,+person))? - Time taken 0.00s]
[:- modeb(100,parent_of(-person,+person))? - Time taken 0.00s]
[:- modeb(100,parent_of(+person,-person))? - Time taken 0.00s]
[:- modeb(100,sister_of(+person,-person))? - Time taken 0.00s]
[Testing for contradictions]
[No contradictions found]
[Generalising aunt_of(jane,henry).]
[Most specific clause is]
aunt_of(A,B) :- parent_of(C,B), sister_of(A,C).
[Learning aunt_of/2 from positive examples]
[C:-0,12,11,0 aunt_of(A,B).]
[C:6,12,4,0 aunt_of(A,B) :- parent_of(C,B).]
[C:6,12,3,0 aunt_of(A,B) :- parent_of(C,B), sister_of(A,C).]
[C:6,12,3,0 aunt_of(A,B) :- parent_of(C,B), sister_of(A,D).]
[C:4,12,6,0 aunt_of(A,B) :- sister_of(A,C).]
[5 explored search nodes]
f=6,p=12,n=3,h=0
[Result of search is]
aunt_of(A,B) :- parent_of(C,B), sister_of(A,C).
[3 redundant clauses retracted]
aunt_of(A,B) :- parent_of(C,B), sister_of(A,C).
[Total number of clauses = 1]
[Time taken 0.00s]

COM3250 / 6170 20 2010-2011

 ILP as a Research Programme

• Progol is just one a set of ILP languages

– > 50 listed at http://www.cs.bris.ac.uk/˜ILPnet2/Systems/
– A popular descendant is Aleph:
(http://web.comlab.ox.ac.uk/oucl/research/areas/machlearn/Aleph/)

• See also:
– http://www.doc.ic.ac.uk/˜shm/ilp.html
– http://en.wikipedia.org/wiki/Inductive_logic_programming

• Applications include learning structural principles underlying protein folding from structural
genomics data (Cootes, Muggleton, and Sternberg: The automatic discovery of structural
principles describing protein fold space. Journal of Molecular Biology, 2003)
– See also http://www.doc.ic.ac.uk/˜shm/applications.html

COM3250 / 6170 21 2010-2011