Learning from Observations

Chapter 18
Section 1 – 3
 Outline
• Learning agents
• Inductive learning
• Decision tree learning
 Learning
• Learning is essential for unknown environments,
– i.e., when designer lacks omniscience

• Learning is useful as a system construction

method,
– i.e., expose the agent to reality rather than trying to
write it down

• Learning modifies the agent's decision

mechanisms to improve performance
Learning agents
 Learning element
• Design of a learning element is affected by
– Which components of the performance element are to
be learned
– What feedback is available to learn these components
– What representation is used for the components

• Type of feedback:
– Supervised learning: correct answers for each
example
– Unsupervised learning: correct answers not given
– Reinforcement learning: occasional rewards
 Inductive learning
• Simplest form: learn a function from examples

f is the target function

An example is a pair (x, f(x))

Problem: find a hypothesis h
such that h ≈ f
given a training set of examples
(This is a highly simplified model of real learning:
– Ignores prior knowledge
– Assumes examples are given)
–
 Inductive learning method
• Construct/adjust h to agree with f on training set
• (h is consistent if it agrees with f on all examples)
• E.g., curve fitting:

•
 Inductive learning method
• Construct/adjust h to agree with f on training set
• (h is consistent if it agrees with f on all examples)
• E.g., curve fitting:
 Inductive learning method
• Construct/adjust h to agree with f on training set
• (h is consistent if it agrees with f on all examples)
• E.g., curve fitting:
 Inductive learning method
• Construct/adjust h to agree with f on training set
• (h is consistent if it agrees with f on all examples)
• E.g., curve fitting:

•
 Inductive learning method
• Construct/adjust h to agree with f on training set
• (h is consistent if it agrees with f on all examples)
• E.g., curve fitting:
 Inductive learning method
• Construct/adjust h to agree with f on training set
• (h is consistent if it agrees with f on all examples)
• E.g., curve fitting:

• Ockham’s razor: prefer the simplest hypothesis

consistent with data
 Learning decision trees
Problem: (Restaurant) to decide whether to wait for a table
at a restaurant, based on the following attributes:
1. Alternate: is there an alternative restaurant nearby?
2. Bar: is there a comfortable bar area to wait in?
3. Fri/Sat: is today Friday or Saturday?
4. Hungry: are we hungry?
5. Patrons: number of people in the restaurant (None, Some, Full)
6. Price: price range ($, $$, $$$)
7. Raining: is it raining outside?
8. Reservation: have we made a reservation?
9. Type: kind of restaurant (French, Italian, Thai, Burger)
10. WaitEstimate: estimated waiting time (0-10, 10-30, 30-60, >60)
 Attribute-based representations
• Examples described by attribute values (Boolean, discrete, continuous)
• E.g., Restaurant – those situations where I will/won't wait for a table:

• Classification of examples is positive (T) or negative (F)

•
 Decision trees
• One possible representation for hypotheses
• E.g., here is the “true” tree for deciding whether to wait:
 Expressiveness
• Decision trees can express any function of the input attributes.
• E.g., for Boolean functions, each truth table row is a path from root to leaf:

• Trivially, there is a consistent decision tree for any training set with one path
to leaf for each example (unless f nondeterministic in x) but it probably won't
generalize to new examples

• Prefer to find more compact decision trees

Hypothesis spaces
 Hypothesis spaces
How many distinct decision trees with n Boolean attributes?
= number of Boolean functions
n
= number of distinct truth tables with 2n rows = 22

• E.g., with 6 Boolean attributes, there are 18,446,744,073,709,551,616

trees

How many purely conjunctive hypotheses (e.g., Hungry  Rain)?

• Each attribute can be in (positive), in (negative), or out (“not sure?”)
 3n distinct conjunctive hypotheses
• More expressive hypothesis space
– increases chance that target function can be expressed
– increases number of hypotheses consistent with training set
 may get worse predictions

– How to arrange the tree (of training set), so that the tree is smaller (to
make its tree-depth minimal and shallow)? => to make the decision
tree as small as possible.
 Decision tree learning
• Aim: find a small tree consistent with the training examples
• Idea: (recursively) choose "most significant" attribute as root of
(sub)tree
 Choosing an attribute
• Aim: find a small tree consistent with the training examples.
• Reason: (recursively) choose “most significant” attribute as
root of (sub)tree
• Idea: a good attribute splits the examples into subsets
that are (ideally) "all positive" or "all negative"

• Patrons? Yes, this is a better choice!

• (“To wait or not” depends more on Patrons or on Type?)
•
 Entrophy & Information
• Information answers questions.
• Scale of Entrophy [0..1]
• Entrophy 1 (answer to Boolean question with prior
<0.5, 0.5>)
• Entrophy 0 (answer to Boolean question with
known result.)
• Information in an answer when prior is:
• <P1, . . ., Pn> is Entrophy I(<P1, . . . , Pn>)
= Σi=1 -Pi log2 Pi
• (also called entropy of the prior)
 Entrophy & Reminder
• <P1, . . ., Pn> is Entrophy I(<P1, . . . , Pn>)
= Σi=1 -Pi log2 Pi
• Suppose we have p positive and n negative
examples at the root.
• Hence I(<p/(p + n), n/(p + n)>) bits needed to
classify a new example.
• For example, for 12 restaurant examples and
p=n=6, we need 1 bit.
 Entrophy & Reminder
• Suppose an attribute A splits the examples E into
subsets Ei, each of which (we hope) needs less
information to complete the classification.
• Let Ei have pi positive and ni negative examples.
Then I(<pi/(pi + ni), ni/(pi + ni)>) bits needed to
classify Ei.
• The remaining bits after choosing A will be
v
p i ni pi ni
remainder ( A)  I( , )
i 1 p  n pi  ni pi  ni

• Idea: Choose A such that Remainder(A) is minimal

(That is, after choosing A, what would be the remaining tree?)
 Using information theory
• Back to the DTL algorithm.
• To implement Choose-Attribute in the DTL
algorithm
• Information Content (Entropy):
I(P(v1), … , P(vn)) = Σi=1 -P(vi) log2 P(vi)
• For a training set containing p positive examples
and n negative examples:

p n p p n n
I( , )  log2  log2
pn pn pn pn pn pn
 Information gain
• A chosen attribute A divides the training set E into subsets
E1, … , Ev according to their values for A, where A has v
distinct values.
v
p i ni pi ni
remainder ( A)  I( , )
i 1 p  n pi  ni pi  ni

• Information Gain (IG) or reduction in entropy from the

attribute test:

p n
IG ( A) I ( , )  remainder ( A)
pn pn
• Choose the attribute with the largest IG
 Information gain
For the training set, p = n = 6, I(6/12, 6/12) = 1 bit
Consider the attributes Patrons and Type (and others too):
2 4 6 2 4
IG( Patrons ) 1  [ I (0,1)  I (1,0)  I ( , )] .0541 bits
12 12 12 6 6
2 1 1 2 1 1 4 2 2 4 2 2
IG(Type ) 1  [ I ( , )  I ( , )  I ( , )  I ( , )] 0 bits
12 2 2 12 2 2 12 4 4 12 4 4

Patrons has the highest IG of all attributes and so is chosen by the DTL
algorithm as the root
p n p p n n
I( , )  log 2  log 2
pn pn pn pn pn pn
 Information gain
For the training set, p = n = 6, I(6/12, 6/12) = 1 bit

Consider the attributes Patrons and Type (and others too):

2 4 6 2 4
IG( Patrons ) 1  [ I (0,1)  I (1,0)  I ( , )] .0541 bits
12 12 12 6 6
2 1 1 2 1 1 4 2 2 4 2 2
IG(Type ) 1  [ I ( , )  I ( , )  I ( , )  I ( , )] 0 bits
12 2 2 12 2 2 12 4 4 12 4 4

Patrons has the highest IG of all attributes and so is chosen by the DTL
algorithm as the root
 Example contd.
• Decision tree learned from the 12 examples:

• Substantially simpler than “true” tree---a more complex

hypothesis isn’t justified by small amount of data (that is,
the problem of overfitting).
 Performance measurement
How do we know that h ≈ f ?
1. Use theorems of computational/statistical learning theory
2. Try h on a new test set of examples
(use same distribution over example space as training set)
Learning curve = % correct on test set as a function of training set size
 Summary
• Learning needed for unknown environments,
lazy designers
• Learning agent = performance element +
learning element
• For supervised learning, the aim is to find a
simple hypothesis approximately consistent with
training examples
• Decision tree learning using information gain
• Learning performance = prediction accuracy
measured on test set