Scribd
Upload
97 views27 pages

CSE543: Machine Learning: Lecture 2: August 6, 2014

This document provides an overview of a machine learning lecture on concept learning. It discusses: 1) An example concept learning task to predict whether Aldo enjoys his favorite water sport based on attributes like weather and temperature. 2) The goal is to learn a hypothesis or "concept" from training examples that best matches the target concept of whether the sport is enjoyed. 3) Possible hypotheses are represented by constraints on attributes, and the FIND-S algorithm is presented to generalize from positive examples to infer the most specific hypothesis. 4) Issues with FIND-S, like its inability to account for negative examples, are noted for potential improvement in future lectures.

Uploaded by

Ranjan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
97 views27 pages

CSE543: Machine Learning: Lecture 2: August 6, 2014

This document provides an overview of a machine learning lecture on concept learning. It discusses: 1) An example concept learning task to predict whether Aldo enjoys his favorite water sport based on attributes like weather and temperature. 2) The goal is to learn a hypothesis or "concept" from training examples that best matches the target concept of whether the sport is enjoyed. 3) Possible hypotheses are represented by constraints on attributes, and the FIND-S algorithm is presented to generalize from positive examples to infer the most specific hypothesis. 4) Issues with FIND-S, like its inability to account for negative examples, are noted for potential improvement in future lectures.

Uploaded by

Ranjan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 27

CSE543: Machine Learning

Lecture 2: August 6, 2014


Example

Labels
3

Example
• Learning Phase
Outloo Play=Ye Play=No Temperature Play=Yes Play=No
k s Hot 2/5
2/9
Sunny 2/9 3/5 Mild
4/9 2/5
Overcas 4/9 0/5
t
Cool
3/9 1/5
Rain 3/9 2/5
Humidity Play=Yes Play=No Wind Play=Yes Play=No
Strong 3/5
High 3/9 4/5 3/9
Weak
Normal 6/9 1/5 6/9 2/5

P(Play=Yes) = 9/14 P(Play=No) = 5/14


• Test Phase

– Given a new instance

x’=(Outlook=Sunny, Temperature=Cool,
Humidity=High, Wind=Strong)

Is Play possible?
• Test Phase

– Given a new instance

x’=(Outlook=Sunny, Temperature=Cool,
Humidity=High, Wind=Strong)

Is Play possible?
This is the task of ML algorithms …

Lets start with CONCEPT LEARNING


Concept Learning
• Concept learning can be formulated as a problem of
searching through a predefined space of potential
hypotheses for the hypothesis that best fits the training
examples.

• Inferring a boolean-valued function from training


examples of its input and output: Learning Concepts
from Examples

• “Concept” typically means categorization based on


features
Concept Learning Task
• Days on which Aldo enjoys his favorite water sport
… we need to learn this task ..
• The task is -> EnjoySport – i.e. Aldo likes his
favorite sport
• Training set
Example Sky AirTemp Humidity Wind Water Forecast Enjoy Sport

1 Sunny Warm Normal Strong Warm Same Yes

2 Sunny Warm High Strong Warm Same Yes

3 Rainy Cold High Strong Warm Change No

4 Sunny Warm High Strong Cool Change Yes


Example Sky AirTemp Humidity Wind Water Forecast Enjoy Sport
1 Sunny Warm Normal Strong Warm Same Yes

Concept Learning Task


2 Sunny Warm High Strong Warm Same Yes
3 Rainy Cold High Strong Warm Change No
4 Sunny Warm High Strong Cool Change Yes

• Target concept:
▫ “Days on which Aldo enjoys his favourite water sport”
▫ Two categories: yes/no

• Some possible concepts:


▫ Enjoy sport when Sky = Sunny
▫ Enjoy sport when Airtemp = Warm
Concept Learning Task
• In Concept learning there is a notion of all possible
hypotheses and best hypothesis

• Lets talk about how we generate hypothesis


▫ “?” represents that any value is acceptable
▫ we specify a single required value
▫  for no value is acceptable

• If some instance x satisfies all the constraints of


hypothesis h, then h classifies x as a positive example
(h(x) = 1)
How do we represent hypotheses
• Sample hypothesis:
▫ H1: Enjoy sport only on cold days with high
humidity
 (?,cold, high, ?,?,?)

▫ H2: Most general hypothesis - everyday is positive


day for game
 (?,?,?,?,?,?)

▫ H3: Most specific possible hypothesis – no day is


good day for game
 (, , , , , )
EnjoySport Concept Learning Task
• Given
▫ Instances X: Possible days, described
by attributes
 Sky (with possible values Sunny, Cloudy, and
Rainy)
 AirTemp (with values Warm and Cold)
 Humidity (with values Normal and High)
 Wind (with values Strong and Weak)
 Water (with values Warm and Cool)
 Forecast (with values Same and Change)
EnjoySport Concept Learning Task

▫ Hypotheses H: Each hypothesis is


described by a conjunction of the
attributes Sky, AirTemp, Humidity,
Wind, Water, and Forecast

▫ The constraints may be "?" (any value


is acceptable), "∅" (no value is
acceptable), or a specific value
EnjoySport Concept Learning Task

• Target concept c: EnjoySport: X→{0,1}

• Training examples D <x, c(x)>: Positive


and negative examples of target function

• For positive examples, c(x) = 1 and for


negative examples, c(x) = 0
EnjoySport Concept Learning Task

• Determine
▫ A hypothesis h such that h(x)=c(x) for
all x in X
EnjoySport Concept Learning Task

 Sky (with possible values Sunny, Cloudy, and Rainy)


 AirTemp (with values Warm and Cold)
 Humidity (with values Normal and High)
 Wind (with values Strong and Weak)
 Water (with values Warm and Cool)
 Forecast (with values Same and Change)

• What is the size of instance space


• How many different hypothesis exist?
▫ Syntactically?
▫ Semantically?
EnjoySport Concept Learning Task

 Sky (with possible values Sunny, Cloudy, and Rainy)


 AirTemp (with values Warm and Cold)
 Humidity (with values Normal and High)
 Wind (with values Strong and Weak)
 Water (with values Warm and Cool)
 Forecast (with values Same and Change)

• What is the size of instance space: 96


• How many different hypothesis exist?:
▫ Syntactically? : 5120
▫ Semantically? : 973
Inductive Learning Hypothesis
• Inductive Learning: Learning by example – the
system tries to induce a general rule by the set
of observed examples

• Inductive Learning Hypothesis: Any hypothesis


found to approximate the target function well
over a sufficiently large set of training
examples will also approximate the target
function well over other unobserved samples
How you create hypothesis from instances
FIND-S Algorithm

• Initialize h to the most specific hypothesis in H


• For each positive training instance x
▫ For each attribute constraint ai in h
▫ If the constraint ai is satisfied by x
▫ Then do nothing
▫ Else replace ai in h by the next more general
constraint that is satisfied by x
• Output hypothesis h
Example Sky AirTemp Humidity Wind Water Forecast EnjoySport
1 Sunny Warm Normal Strong Warm Same Yes
2 Sunny Warm High Strong Warm Same Yes
3 Rainy Cold High Strong Warm Change No
4 Sunny Warm High Strong Cool Change Yes

• Step1: h<-- (, , , , , )

• Upon observing the first training example, current


hypothesis is too specific. So replace by the next
more general constraint

• h<--<Sunny,Warm,Normal,Strong,Warm,Same>
Example Sky AirTemp Humidity Wind Water Forecast EnjoySport
1 Sunny Warm Normal Strong Warm Same Yes
2 Sunny Warm High Strong Warm Same Yes
3 Rainy Cold High Strong Warm Change No
4 Sunny Warm High Strong Cool Change Yes

• After observing the second training sample, the


hypothesis becomes:
h<--<Sunny,Warm,?,Strong,Warm, Same>

• Ignore the third sample (which is negative)

• After the fourth sample


h<--<Sunny,Warm,?,Strong,?>
Find S Algorithm
Hypothesis found by FIND-S

• h<--<Sunny,Warm,?,Strong,?>

Example Sky AirTemp Humidity Wind Water Forecast EnjoySport

1 Sunny Warm Normal Strong Warm Same Yes

2 Sunny Warm High Strong Warm Same Yes

3 Rainy Cold High Strong Warm Change No

4 Sunny Warm High Strong Cool Change Yes


Any other hypothesis for same data?
• There are other hypotheses consistent with the
data, e.g., <Sunny,?,?,Strong,?,?>
Example Sky AirTemp Humidity Wind Water Forecast EnjoySport

1 Sunny Warm Normal Strong Warm Same Yes

2 Sunny Warm High Strong Warm Same Yes

3 Rainy Cold High Strong Warm Change No

4 Sunny Warm High Strong Cool Change Yes

Find-S finds the most specific hypothesis in H which is consistent


with the positive examples
Major drawback of FIND-S

• No negative training
• What happens with a negative test case?
Major drawback of FIND-S

• No negative training
• What happens with a negative test case?

Can we come up with a better


algorithm?
Next Class:

Version Spaces and The Candidate Elimination


Algorithm

You might also like