INTRODUCTION TO

MACHINE LEARNING

1
 Vaageswari College of Engineering
Department of CSE

History of Machine Learning

2
Vaageswari College of Engineering
Department of CSE

3
 Vaageswari College of Engineering
Department of CSE

Factors behind Popularity of ML

4
 Vaageswari College of Engineering
Department of CSE

Learning ?

Definition

• “ Learning is the ability to improve the behavior on

experience”

5
 Vaageswari College of Engineering
Department of CSE

Definitions of Machine Learning

• Machine learning is an application of artificial
intelligence (AI) that provides systems the ability to
automatically learn and improve from experience
without being explicitly programmed.

• Machine Learning is the study of computer

algorithms that improves automatically through
experience

• Machine learning is a subset of AI, which enables the

machine to automatically learn from data, improve
performance from past experiences, and make
predictions
6
 Vaageswari College of Engineering
Department of CSE

• Machine learning contains a set of algorithms that

work on a huge amount of data. Data is fed to these
algorithms to train them, and on the basis of training,
they build the model & perform a specific task.

• The primary aim is to allow the computers learn

automatically without human intervention or
assistance and adjust actions accordingly.

7
 Vaageswari College of Engineering
Department of CSE

Difference between AI, ML & DL

8
 Vaageswari College of Engineering
Department of CSE

Types of Machine Learning

Supervised Machine Learning :
In supervised learning technique, we train the
machines using the "labelled" dataset, and based on
the training, the machine predicts the output.

Un-Supervised Machine Learning :

In unsupervised machine learning, the machine is
trained using the unlabeled dataset, and the machine
predicts the output without any supervision.

9
 Vaageswari College of Engineering
Department of CSE

Semi-Supervised Machine learning

Semi-Supervised Machine learning uses the

combination of labelled and unlabeled datasets to train
a machine

Reinforcement learning
Reinforcement learning works on a feedback-based
process, in which an AI agent (A software component)
automatically explore its surrounding by hitting & trail,
taking action, learning from experiences, and improving
its performance. 10
 Vaageswari College of Engineering
Department of CSE

WELL-POSED LEARNING PROBLEMS

• Definition of Machine Learning (by Tom.Mitchel)

A computer program is said to learn from experience E with

respect to some class of tasks T and performance measure
P, if its performance at tasks in T, as measured by P, improves
with experience E.

11
 Vaageswari College of Engineering
Department of CSE

• Machine Learning?
Improve over task T.
With respect to performance measure P.
Based on experience E.

What are T, P, E ?
How do we formulate a machine learning problem?

12
 Vaageswari College of Engineering
Department of CSE

A checkers learning problem

Task T:
Playing checkers
Performance measure P:
Percent of games won against opponents
Training experience E:
Playing practice games against itself

13
 Vaageswari College of Engineering
Department of CSE

A handwriting recognition learning problem:

• Task T: recognizing and classifying handwritten
words within images
• Performance measure P: percent of words correctly
classified
• Training experience E: a database of handwritten
words with given classifications

14
 Vaageswari College of Engineering
Department of CSE

A robot driving learning problem:

• Task T: driving on public four-lane highways using vision
sensors
• Performance measure P: average distance traveled before
an error (as judged by human overseer)
• Training experience E: a sequence of images and steering
commands recorded while observing a human driver

15
 Some disciplines Vaageswari College of Engineering
Department of CSE

& their influence on Machine Learning.

 Artificial intelligence
 Bayesian methods
 Computational complexity theory
 Statistics
 Psychology and neurobiology
 Philosophy
 Information theory
 Neural Networks
 Data Mining

16
 Vaageswari College of Engineering
Department of CSE

Artificial intelligence: Learning as an approach to improving

problem solving. Using prior knowledge together with training
data to guide learning.
Bayesian methods: Bayes' theorem as the basis for calculating
probabilities of hypotheses. The naive Bayes classifier.
Computational complexity theory: Theoretical bounds on the
inherent complexity of different learning tasks, measured in
terms of the computational effort, number of training examples,
number of mistakes.
Statistics: Characterization of errors (e.g., bias and variance) that
occur when estimating the accuracy of a hypothesis based on a
limited sample of data. Confidence intervals, statistical tests.
• .
17
 Vaageswari College of Engineering
Department of CSE

Psychology and neurobiology : Study of Human behaviour,

Neurobiological studies motivating artificial neural network
models of learning.
Philosophy: Analysis of the justification for generalizing beyond
observed data.
Information theory: Measures of entropy and information
content, relationships among training sequences for encoding a
hypothesis.
data mining: data mining is defined as a process used to extract
usable data from a larger set of any raw data. Data mining involves
exploring and analyzing large blocks of information to glean
meaningful patterns and trends.
ANN: The field which attempts to mimic the network of neurons
makes up a human brain so that computers will have an option18to
understand things and make decisions in a human-like manner
 Vaageswari College of Engineering
Department of CSE

DESIGNING A LEARNING SYSTEM

19
 Vaageswari College of Engineering
Department of CSE

Steps in designing learning System

1. Choosing the Training Experience

2. Choosing the Target Function
3. Choosing a Representation for the Target Function
4. Choosing a Function Approximation Algorithm

20
 Vaageswari College of Engineering
Department of CSE

Choosing the Training Experience

• choose the type of training experience from which our

system will learn
• One key attribute is whether the training experience
provides direct or indirect feedback regarding the choices
made by the performance system
• A second important attribute of the training experience is
the degree to which the learner controls the sequence of
training examples.
• A third important attribute of the training experience is
how well it represents the distribution of examples over
which the final system performance P must be measured.
21
 Vaageswari College of Engineering
Department of CSE

Choosing the Target Function

• determine exactly what type of knowledge will be

learned and how this will be used by the performance
program.
• Let us call this target function V
• Target function V for checkers game
1. if b is a final board state that is won, then V(b) = 100
2. if b is a final board state that is lost, then V(b) = -100
3. if b is a final board state that is drawn, then V(b) = 0

22
 Vaageswari College of Engineering
Department of CSE

Choosing a Representation for the Target Function

We have following design choices for representing target

function:
• We could, for example, allow the program to represent
using a large table with a distinct entry specifying the
value for each distinct board state
• Or we could allow it to represent using a collection of
rules that match against features of the board state, or a
quadratic polynomial function of predefined board
features
• or an artificial neural network.
23
 Vaageswari College of Engineering
Department of CSE

Choosing a Function Approximation Algorithm

Choose the learning algorithm to infer the target function
design choices we have like
1. linear programming,
2. Gradient descent
EX: LMS(least mean squares)
1. ESTIMATING TRAINING VALUES
2. ADJUSTING THE WEIGHTS

24
 Vaageswari College of Engineering
Choices in designing the Department of CSE

Checkers Learning program.

25
 Vaageswari College of Engineering
Department of CSE

Issues in Machine Learning

• How much training data is sufficient?
• Can prior knowledge be helpful even when it is only
approximately correct?
• What is the best strategy for choosing a useful next
training experience
• What is the best way to reduce the learning task to
one or more function approximation problems.
• How can the learner automatically alter its
representation to improve its ability to represent and
learn the target function

26
 Vaageswari College of Engineering
Department of CSE

Applications of Machine Learning

27
 Vaageswari College of Engineering
Department of CSE

1. Image Recognition:
Image recognition is used to identify objects, persons, places,
digital images, etc. The popular use case of image recognition and
face detection. Ex: Face Recognition System
2. Speech Recognition:
Speech recognition is a process of converting voice instructions
into text, and it is also known as "Speech to text", or "Computer
speech recognition. Ex: Google Speech recognition, Ex:
SPHINX system
3. Traffic prediction
It predicts the traffic conditions such as whether traffic is cleared,
slow-moving, or heavily congested with the help of two ways.
Ex: Using Google Maps for Traffic Predictions 28
 Vaageswari College of Engineering
Department of CSE

4. Product Recommendations
Machine learning is widely used by various e-commerce companies
such as Amazon, Flipkart, for product recommendation to the user.
5. Self Driving Cars
One of the most exciting applications of machine learning is self-
driving cars. Machine learning plays a significant role in self-driving
cars. Ex: Learning to drive an autonomous vehicle( ALVINN system)
6. Email Spam and Malware Filtering
Whenever we receive a new email, it is filtered automatically as
important, normal, and spam
7. Personal Assistants
Personal Assistants help us in finding the information using our voice
instruction. These assistants can help us in various ways just by our
29
voice instructions such as Play music, call someone. Ex: Siri, Alexa
 Vaageswari College of Engineering
Department of CSE

8. Online Fraud Detection

Machine learning is making our online transaction safe and secure
by detecting fraud transaction.
9. Stock Market trading
Machine learning is widely used in stock market trading. In the
stock market, there is always a risk of up and downs in shares, so
for this machine learning's long short term memory neural
network is used for the prediction of stock market trends.
10. Medical Diagnosis
machine learning is used for diseases diagnoses.
11. Automatic Language Translation
machine learning helps us by converting the text into our known
languages. Ex: Google's GNMT (Google Neural Machine
30
Translation)
 Vaageswari College of Engineering
Department of CSE

Concept Learning

31
32
Vaageswari College of Engineering
Department of CSE

33
 Vaageswari College of Engineering
Department of CSE

CONCEPT LEARNING TASK

34
Vaageswari College of Engineering
Department of CSE

35
Vaageswari College of Engineering
Department of CSE

36
Vaageswari College of Engineering
Department of CSE

37
Vaageswari College of Engineering
Department of CSE

38
Vaageswari College of Engineering
Department of CSE

39
 Vaageswari College of Engineering
Department of CSE

CONCEPT LEARNING AS SEARCH

• Concept learning can be viewed as the task of searching through a
large space of hypotheses implicitly defined by the hypothesis
representation.

• The goal of this search is to find the hypothesis that best fits the
training examples.

Example:
The instances X and hypotheses H in the EnjoySport learning task.

40
 Vaageswari College of Engineering
Department of CSE

• Given that the attribute Sky has three possible values, and that
AirTemp, Humidity, Wind, Water, and Forecast each have two possible
values, the instance space X contains exactly 3 .2 2 .2 2 .2 = 96
distinct instances.
• A similar calculation shows that there are 5.4.4 .4 .4.4 = 5 120
Syntactically distinct hypotheses within H
• Every hypothesis containing one or more “ ø" symbols
represents the empty set of instances; that is, it classifies
every instance as negative.

41
 Vaageswari College of Engineering
General-to-Specific Department of CSE

Ordering of Hypotheses

• Many algorithms for concept learning organize the search through the
hypothesis space by relying on a very useful structure that exists for any
concept learning problem: a general-to-specific ordering of hypotheses.
• To illustrate the general-to-specific ordering, consider the two
hypotheses
• h1 = (Sunny, ?, ?, Strong, ?, ?)
• h2 = (Sunny, ?, ?, ?, ?, ?)
• Now consider the sets of instances that are classified positive by hl and
by h2
• Because h2 imposes fewer constraints on the instance, it classifies more
instances as positive. In fact, any instance classified positive by hl will
also be classified positive by h2. Therefore, we say that h2 is more
general than h1

42
 Vaageswari College of Engineering
Department of CSE

• for any instance x in X and hypothesis h in H, we say that x satifies h if

and only if h(x) = 1.
• We now define the more-general-than-or-equal-to in terms of the
sets of instances that satisfy
• the two hypotheses: Given hypotheses hj and hk, hj is more-general-
than- equal-to hk if and only if any instance that satisfies hk also
satisfies hi.

43
Vaageswari College of Engineering
Department of CSE

44
 Vaageswari College of Engineering
Department of CSE

• Consider the three hypotheses hl, h2, and h3 from our Enjoysport
example, shown in Figure.

• hypothesis h2 is more general than h1 because every instance that

satisfies h1 also satisfies h2. Similarly, h2 is more general than h3
• neither h1 nor h3 is more general than the other; although the
instances satisfied by these two hypotheses intersect, neither set
subsumes the other.

45
 Vaageswari College of Engineering
FIND-S Algorithm Department of CSE

• Concept learning consist of two types of hypothesis

• General Hypothesis
• Specific Hypothesis

• G={ ‘?’, ‘?’, ‘?’, ‘?’ - - - , ‘?’ }

• S= {‘𝚽’, ‘𝚽’,‘𝚽’, ‘𝚽’, --- ‘𝚽’}
• Find –S Algorithm will always starts with most
specific hypothesis.
• Find –S algorithm only considers positive examples

46
 Vaageswari College of Engineering
Department of CSE

• FIND-S Algorithm
1. Initialize h to most specific hypothesis
h= {‘𝚽’, ‘𝚽’,‘𝚽’, ‘𝚽’, --- ‘𝚽’, }

2. For each positive Example:

For each attribute in the example:
if Attribute value = Hypothesis value:
Do Nothing.
else:
Replace the hypothesis value with more general
constraint ‘?’

47
 Vaageswari College of Engineering
Department of CSE

FIND-S Algorithm

48
Vaageswari College of Engineering
Department of CSE

49
 Vaageswari College of Engineering
Department of CSE

• The first step of FINDS is to initialize h to the most specific hypothesis in H

h= {‘𝚽’, ‘𝚽’,‘𝚽’, ‘𝚽’, --- ‘𝚽’, }

• Upon observing the first training example from Table, which happens to be a
positive example, it becomes clear that our hypothesis is too specific.

• so each is replaced by the next more general constraint {hat fits the example;
namely, the attribute values for this training example.
h-> (Sunny, Warm, Normal, Strong, Warm, Same)
This h is still very specific

• the second training example (also positive in this case) forces the algorithm to
further generalize h, this time substituting a "?' in place of any attribute value in
h that is not satisfied by the new example.

• The refined hypothesis in this case is

h -> (Sunny, Warm, ?, Strong, Warm, Same ) 50
 Vaageswari College of Engineering
Department of CSE

• Upon encountering the third training example-in this case a negative

example- the algorithm makes no change to h. In fact, the FIND-S
algorithm simply ignores every negative example
• To complete our trace of FIND-S, the fourth (positive) example leads to
a further generalization of h
h <- (Sunny, Warm, ?, Strong, ?, ?)

51
Vaageswari College of Engineering
Department of CSE

52
Exercise

53
VERSION SPACES & Vaageswari College of Engineering
Department of CSE
CANDIDATE-ELIMINATION
ALGORITHM

54
 Vaageswari College of Engineering
Department of CSE

Version Space:

S= {‘𝚽’, ‘𝚽’,‘𝚽’, ‘𝚽’, --- ‘𝚽’ } G={ ‘?’, ‘?’, ‘?’, --- ‘?’}

S= {‘𝚽’, ‘𝚽’,‘𝚽’, ‘𝚽’, --- ‘𝚽’ }

G={ ‘?’, ‘?’, ‘?’, --- ‘?’}

55
Vaageswari College of Engineering
Vaageswari College
Department of of
CSEEngineerin
Department of CSE

56
 Vaageswari College of Engineering
Department of CSE

CANDIDATE ELIMINATION Algorithm

1. Uses version Space

2. Considers both Positive and Negative examples.
3. We have both Specific and General Hypothesis
4. For a Positive Hypothesis we tend to generate
specific Hypothesis.
5. For Negative example we tend to make general
hypothesis more specific

S= {‘𝚽’, ‘𝚽’,‘𝚽’, ‘𝚽’, --- ‘𝚽’ }

G={ ‘?’, ‘?’, ‘?’, --- ‘?’}

57
 Vaageswari College of Engineering
Department of CSE

Algorithm
1. Initialize G and S as most general and specific
hypothesis.
2. For each example e:
if e is Positive:
Make specific hypothesis More general
else:
Make general hypothesis More specific

58
 Vaageswari College of Engineering
Department of CSE

DATA SET:
CONCEPT: Days on which person enjoys Water sport

Step-1: S0= {‘𝚽’, ‘𝚽’,‘𝚽’, ‘𝚽’, --- ‘𝚽’ }

G0={ ‘?’, ‘?’, ‘?’, --- ‘?’}

Step-2: S1= {Sunny, Warm, Normal, Strong, Warm, Sam}

G1={ ‘?’, ‘?’, ‘?’, --- ‘?’}

Step-3: S2= {Sunny, Warm, ?, Strong, Warm, Sam}

G2={ ‘?’, ‘?’, ‘?’, --- ‘?’} 59
 Vaageswari College of Engineering
Department of CSE

Step-4:
S3= {Sunny, Warm, ?, Strong, Warm, Sam}

G3= {< ‘Sunny’, ‘?’, ‘?’, ‘?’, ‘?’, ‘?’>, < ‘?’, ‘Warm’, ‘?’, ‘?’, ‘?’, ‘?’>, < ‘?’, ‘?’, ‘Normal’, ‘?’, ‘?’,‘?’>
< ‘?’, ‘?’, ‘?’, ‘?’, ‘Cool’, ‘?’>, < ‘?’, ‘?’, ‘?’, ‘?’, ‘?’, ‘Same’>}

G3= { < ‘Sunny’, ‘?’, ‘?’, ‘?’, ‘?’, ‘?’>, < ‘?’, ‘Warm’, ‘?’, ‘?’, ‘?’, ‘?’, >, < ‘?’, ‘?’, ‘?’, ‘?’, ‘?’, ‘Same’>}

Step-5:
S4= {Sunny, Warm, ?, Strong, ?, ?}
G4= { < ‘Sunny’, ‘?’, ‘?’, ‘?’, ‘?’, ‘?’>, < ‘?’, ‘Warm’, ‘?’, ‘?’, ‘?’, ‘?’>}
60
 Vaageswari College of Engineering
Department of CSE

S4= {Sunny, Warm, ?, Strong, ?, ?}

< ‘Sunny’, ‘?’, ‘?’, ‘Strong’, ‘?’, ‘?’, > < ‘Sunny’, ‘Warm’, ‘?’, ‘?’, ‘?’, ‘?’> < ‘?’, ‘Warm’, ‘?’, ‘?’, ‘Strong’, ‘?’, ‘?’>

G4= { < ‘Sunny’, ‘?’, ‘?’, ‘?’, ‘?’, ‘?’, ‘?’>, < ‘?’, ‘Warm’, ‘?’, ‘?’, ‘?’, ‘?’, ‘?’>}

61
 Vaageswari College of Engineering
Department of CSE

Decision Tree Learning

62
 Vaageswari College of Engineering
Department of CSE

Definition:

63
Vaageswari College of Engineering
Department of CSE

64
 Vaageswari College of Engineering
Department of CSE

ID3 Algorithm:

65
 Vaageswari College of Engineering
Department of CSE

Data Set:

66
Vaageswari College of Engineering
Department of CSE

67
Vaageswari College of Engineering
Department of CSE

68
Vaageswari College of Engineering
Department of CSE

69
Vaageswari College of Engineering
Department of CSE

70
Vaageswari College of Engineering
Department of CSE

71
Vaageswari College of Engineering
Department of CSE

72
Vaageswari College of Engineering
Department of CSE

73
Vaageswari College of Engineering
Department of CSE

74
Vaageswari College of Engineering
Department of CSE

75
Vaageswari College of Engineering
Department of CSE

76
Vaageswari College of Engineering
Department of CSE

77
Vaageswari College of Engineering
Department of CSE

78
Vaageswari College of Engineering
Department of CSE

79
Vaageswari College of Engineering
Department of CSE

80
Vaageswari College of Engineering
Department of CSE

81
Vaageswari College of Engineering
Department of CSE

82
Vaageswari College of Engineering
Department of CSE

83
Vaageswari College of Engineering
Department of CSE

84
Vaageswari College of Engineering
Department of CSE

85
Vaageswari College of Engineering
Department of CSE

86
Vaageswari College of Engineering
Department of CSE

87
Vaageswari College of Engineering
Department of CSE

88
Vaageswari College of Engineering
Department of CSE

89
Vaageswari College of Engineering
Department of CSE

90
Vaageswari College of Engineering
Department of CSE

91
Vaageswari College of Engineering
Department of CSE

92
Vaageswari College of Engineering
Department of CSE

93
Vaageswari College of Engineering
Department of CSE

94
Vaageswari College of Engineering
Department of CSE

95
Vaageswari College of Engineering
Department of CSE

96
 Vaageswari College of Engineering
Department of CSE

97
 Vaageswari College of Engineering
Alternative Measures for Department of CSE

Selecting Attributes
• One way to avoid this difficulty is to select decision
attributes based on some measure other than
information gain
• One alternative measure that has been used
successfully is the gain ratio
• split information, is sensitive to how broadly and
uniformly the attribute splits the data

98
Handling Training Example Vaageswari College of Engineering
Department of CSE

with Missing Attribute Values

• In certain cases, the available data may be missing
values for some attributes
• One strategy for dealing with the missing attribute
value is to assign it the value that is most common
among training examples at node n.
• A second, more complex procedure is to assign a
probability to each of the possible values of A rather
than simply assigning the most common value to A(x)

99
 Vaageswari College of Engineering
Handling Attributes with Department of CSE

Differing Costs
• In some learning tasks the instance attributes may
have associated costs.
• ID3 can be modified to take into account attribute
costs by introducing a cost term into the attribute
selection measure.
• we might divide the Gain by the cost of the attribute,
so that lower-cost attributes would be preferred
• While such cost-sensitive measures do not guarantee
finding an optimal cost-sensitive decision tree, they
do bias the search in favor of low-cost attributes

100