Machine Learning KTU Module 1
Machine Learning KTU Module 1
2
Machine Learning is...
3
What is Machine Learning
4
What is Machine Learning Cont..
5
Why “Learn” ?
▷ There is no need to “learn” to calculate payroll
▷ Learning is used when:
○ Human expertise does not exist (navigating on Mars),
○ Humans are unable to explain their expertise (speech
recognition)
○ Solution changes in time (routing on a computer
network)
○ Solution needs to be adapted to particular cases (user
biometrics)
8
Data Mining
▷ Retail: Market basket analysis, Customer relationship
management (CRM)
▷ Finance: Credit scoring, fraud detection
▷ Manufacturing: Control, robotics, troubleshooting
▷ Medicine: Medical diagnosis
▷ Telecommunications: Spam filters, intrusion detection
▷ Web mining: Search engines
▷ ...
9
GENERAL CLASSES OF MACHINE
LEARNING PROBLEMS
▷ Learning associations
○ Association rule learning is a machine learning
method for discovering interesting relations,
called “association rules”, between variables in
large databases using some measures of
“interestingness”.
○ How association rules are made use of: Consider
an association rule of the form
X => Y,
that is, if people buy X then they are also likely to
buy Y .
10
Learning associations
11
What We Talk About When We Talk
About “Learning”
▷ Learning general models from a data of particular
examples
▷ Data is cheap and abundant (data warehouses,
data marts); knowledge is expensive and scarce.
▷ Example in retail: Customer transactions to
consumer behavior:
○ People who bought book A also bought book B
▷ Build a model that is a good and useful
approximation to the data.
8
Learning associations
13
GENERAL CLASSES OF MACHINE
LEARNING PROBLEMS
▷ Classification
○ the problem of identifying to which of a set of
categories a new observation belongs, based on a
training set of data containing observations (or
instances) whose category membership is known.
14
Data Mining
▷ Retail: Market basket analysis, Customer relationship
management (CRM)
▷ Finance: Credit scoring, fraud detection
▷ Manufacturing: Control, robotics, troubleshooting
▷ Medicine: Medical diagnosis
▷ Telecommunications: Spam filters, intrusion detection
▷ Web mining: Search engines
▷ ...
9
Classification
16
GENERAL CLASSES OF MACHINE
LEARNING PROBLEMS
▷ Regression
○ the problem of predicting the value of a numeric
variable based on observed values of the variable.
17
Regression
18
Regression
19
Regression
20
Kinds of Machine Learning
▷ Supervised Learning
○ Classification
○ Regression
▷ Unsupervised Learning
▷ Reinforcement Learning
21
Supervised Learning
22
Supervised Learning
23
Supervised Learning
24
Supervised Learning
25
Exercise 1
26
Supervised Learning
FRUIT
No. SIZE COLOR SHAPE
NAME
27
Supervised Learning
FRUIT
No. SIZE COLOR SHAPE
NAME
Rounded shape with
1 Big Red Apple
depression at the top
Heart-shaped to nearly
2 Small Red Cherry
globular
3 Big Green Long curving cylinder Banana
Round to oval,Bunch shape
4 Small Green Grape
Cylindrical
28
Unsupervised Learning
29
Unsupervised Learning
30
Unsupervised Learning
32
Reinforcement Learning
34
Reinforcement Learning Cont..
26
Types of Machine Learning
Machine
Learning
37
Input representation
38
Hypothesis
39
Hypothesis
40
Hypothesis
41
Hypothesis - Example
43
Consider a situation with four binary
variables x1, x2, x3, x4 and one binary
output variable y. What is the size of
the hypothesis space?
44
Reinforcement Learning
34
Reinforcement Learning Cont..
43
Scatter plot of price-power data
(hollow circles indicate positive examples and solid dots
indicate negative examples)
49
The version space consists of hypotheses corresponding to axis-
aligned rectangles contained in the shaded region
51
Vapnik-Chervonenkis (VC) Dimension
52
Vapnik-Chervonenkis (VC) Dimension
53
Vapnik-Chervonenkis (VC) Dimension
Total Data Points = 3
54
Version space
The version space for the problem with respect to the set
D and the space H is the set of hypotheses from H consistent
with D; that is, it is the set
VS D,H = {h ∈ H : h(x) = c(x) for all x ∈ D}
45
Version space
47
Vapnik-Chervonenkis (VC) Dimension
58
Vapnik-Chervonenkis (VC) Dimension
59
Vapnik-Chervonenkis (VC) Dimension
60
Vapnik-Chervonenkis (VC) Dimension
Total Data Points = 4
61
Vapnik-Chervonenkis (VC) Dimension
62
Vapnik-Chervonenkis dimension (VC
dimension)
63
Shattering of a set
▷ Let D be a dataset containing N examples for a
binary classification problem with class labels
0 and 1.
▷ Let H be a hypothesis space for the problem.
▷ Each hypothesis h in H partitions D into two
disjoint subsets as follows:
65
Example
66
▷ Let D be a subset of X containing only a single
number, say, D = {3.5}.
▷ There are 2 dichotomies for this set.
▷ These correspond to the following assignment
of class labels:
67
▷ h4 ∈ H is consistent with the former
dichotomy and h3 ∈ H is consistent with the
latter.
▷ So, to every dichotomy in D there is a
hypothesis in H consistent with the
dichotomy.
▷ Therefore, the set D is shattered by the
hypothesis space H.
68
▷ Let D be a subset of X containing two
elements, say, D = {3.25; 4.75}.
69
▷ In these dichotomies,
○ h1 is consistent with (a),
○ h2 is consistent with (b) and
○ h3 is consistent with (d).
○ But there is no hypothesis hm Є H consistent with
(c).
▷ Thus the two-element set D is not shattered
by H.
▷ the size of the largest finite subset of X
shattered by H is 1. This number is the VC
dimension of H. 70
VC Dimension
▷ VC –dim(constant)=0
▷ VC-dim(single-parametric threshold classifier)=1
▷ VC-dim(intervals) = 2
▷ VC-dim(line) = 3
▷ VC-dim(Axis-aligned rectangles) = 4
71
An axis aligned rectangle can shatter 4
points
72
Vapnik-Chervonenkis (VC) Dimension
52
Vapnik-Chervonenkis (VC) Dimension
53
Vapnik-Chervonenkis (VC) Dimension
Total Data Points = 3
54