1|Page

KRISHNA ENGINEERING COLLEGE

(Approved by AICTE & Affiliated to Dr. APJ Abdul Kalam Technical University (Formerly UPTU), Lucknow)
Department of CSE-Artificial Intelligence
Department of CSE-Artificial Intelligence & Machine Learning

Unit-1: INTRODUCTION

1.1. Learning
1.2. Types of Learning
1.3. Well defined learning problems
1.4. Designing a Learning System
1.5. History of ML
1.6. Introduction of Machine Learning Approaches:
1.6.1. Artificial Neural Network, (Unit-4)
1.6.2. Clustering,
1.6.3. Reinforcement Learning, (Unit-5)
1.6.4. Decision Tree Learning, (Unit-3)
1.6.5. Bayesian networks, (Unit-2)
1.6.6. Support Vector Machine, (Unit-2)
1.6.7. Genetic Algorithm (Unit-5)
1.6.8. Issues in Machine Learning and Data Science Vs Machine Learning;

---------------------------------------------------------------------------------------------------------------------

1.1. Learning (in ppt slides)

1.2. Types of Learning (in ppt slides)
2|Page

1.3. Well defined learning problems

A computer program is said to learn from experience E with respect to some class of tasks T and
performance measure P, if its performance at tasks in T, as measured by P, improves with
experience E
To have a well-defined learning problem, we must identity these three features:
 the class of tasks (T)
 the measure of performance to be improved (P), and
 the source of experience (E)

Example 1: A checkers learning problem:

 Task T: playing checkers
 Performance measure P: percent of games won against opponents
 Training experience E: playing practice games against itself

Example 2: A handwriting recognition learning problem:

 Task T: recognizing and classifying handwritten words within images
 Performance measure P: percent of words correctly classified
 Training experience E: a database of handwritten words with given classifications

Example 3: A robot driving learning problem:

 Task T: driving on public four-lane highways using vision sensors
 Performance measure P: average distance traveled before an error (as judged by human
supervisor)
 Training experience E: a sequence of images and navigation commands recorded while
observing a human driver

Example 4: Text Classification Problem:

 Task T: assign a document to its content category
 Performance measure P: Precision and Recall
 Training experience E: Example pre-classified documents
3|Page

1.4. Designing a Learning System

a) Choosing the Training Experience
1) direct or indirect feedback
2) degree
3) distribution of examples

One key attribute is whether the training experience provides direct or indirect feedback for the
choices made by the performance system.
For example, in learning to play checkers, the system might learn from direct training examples
consisting of individual checkers board states and the correct move for each. Alternatively, it
might have available only indirect information consisting of the move sequences and final
outcomes of various games played.
A second important attribute of the training experience is the degree to which the learner
controls the sequence of training examples i.e. determining the degree to which each move in
the sequence deserves credit or blame for the final outcome.
A third important attribute of the training experience is how well it represents the distribution
of examples over which the final system performance P must be measured.
In general, learning is most reliable when the training examples follow a distribution similar to
that of future test examples.
In our checkers learning scenario, the performance metric P is the percent of games the
system wins in the world tournament.
If its training experience E consists only of games played against itself, there is an obvious
danger that this training experience might not be fully representative of the distribution of
situations over which it will later be tested.
To proceed with our design, let us decide that our system will train by playing games against
itself.
This has the advantage that no external trainer need be present, and it therefore allows the
system to generate as much training data as time permits.
We now have a fully specified learning task.
A checkers learning problem:
 Task T: playing checkers
 Performance measure P:percent of games won in the world tournament
 Training experience E: games played against itself
4|Page

b) Choosing the Target Function

The next design choice is to determine exactly what type of knowledge will be learned and how
this will be used by the performance program
Target function V use the notation V: B → R
Here V maps any legal board state from the set B to some real value R. This target function V
will be assigned higher scores for better board states. Let us therefore define the target value
V( b ) for an arbitrary board state b in B , as follows:
1. if b is a final board state that is won, then V ( b )= 100
2. if b is a final board state that is lost, then V ( b )= -100
3. if b is a final board state that is drawn, then V ( b )= 0
4. if b is a not a final state in the game, then V(b) = V(b'),
where b' is the best final board state that can be achieved starting from b and playing
optimally until the end of the game.
The above is nonoperational definition because this definition is not efficiently computable by
our checkers playing program. The goal of learning in this case is to discover an operational
description of V. The process of learning the target function is often called function
approximation
c) Choosing a Representation for the Target Function
For example, for any given board state, the function 𝑉̂ will be calculated as a linear combination
of the following board features:
xl: the number of black pieces on the board
x2: the number of red pieces on the board
x3: the number of black kings on the board
x4: the number of red kings on the board
x5: the number of black pieces threatened by red
x6: the number of red pieces threatened by black

Thus, our learning program will represent 𝑉̂ (b) as a linear function of the form

Where w0 through w6 are numerical coefficients, or weights, to be chosen by the learning algorithm.
The weights wl through w6 will determine the relative importance of the various board features in
determining the value of the board, whereas the weight w0 will provide an additive constant to the
board value.
5|Page

Partial design of A checkers learning problem:

 Task T: playing checkers
 Performance measure P:percent of games won in the world tournament
 Training experience E: games played against itself
 Target function representation:

d) Choosing a Function Approximation Algorithm

In order to learn the target function 𝑉̂ we require a set of training examples, each describing a
specific board state b and the training value Vtrain(b) for b i.e. (b, Vtrain(b)).
For instance, the following training example describes a board state b in which black has won
the game, x2 = 0 indicates that red has no remaining pieces for which the target function value
Vtrain(b) is therefore +100.

i. Estimating training values

One simple approach is to assign the training value Vtrain(b) to any intermediate board
state b to be 𝑉̂ (Successor(b))
Where
 𝑉̂ is the learner's current approximation to V
 Successor (b) denotes the next board state following b for which it is again the
program's turn to move.

This rule for estimating training values can be summarized as

ii. Adjusting the weights

To specify the learning algorithm for choosing the weights wi to best fit the set of
training examples {(b,Vtrain(b))}.
As a first step we must define what we mean by the best fit to the training data.
One common approach is to define the best hypothesis, or set of weights, which
minimizes the squared error E between the training values and the values predicted by
the hypothesis 𝑉̂ .
6|Page

e) The Final Design

Final design of the checkers learning program

The final design of our checkers learning system can be described by four distinct program
modules. These four modules are:
The Performance System is the module that must solve the given performance task, in this case
playing checkers, by using the learned target function(s). It takes an instance of a new problem
(new game) as input and produces a trace of its solution (game history) as output.
The Critic takes as input the history or trace of the game and produces as output a set of training
examples of the target function.
As shown in the diagram, each training example in this case corresponds to some game state in the
trace, along with an estimate Vtrain, of the target function value for this example
7|Page

The Generalizer takes as input the training examples and produces an output hypothesis that is its
estimate of the target function
In our example, the Generalizer corresponds to the LMS algorithm, and the output hypothesis is
the function f described by the learned weights w0 , ……, w6.
The Experiment Generator takes as input the current hypothesis (currently learned function) and
outputs a new problem (i.e., initial board state) for the Performance System to explore.
Its role is to pick new practice problems that will maximize the learning rate of the overall system.
In our example, the Experiment Generator follows a very simple strategy: It always proposes the
same initial game board to begin a new game. More sophisticated strategies could involve creating
board positions designed to explore particular regions of the state space.

Summary of choices in designing the checkers learning program

References:
https://www.youtube.com/watch?app=desktop&v=GccPUKGl4bA
Tom M. Mitchell, ―Machine Learning, McGraw-Hill Education (India) Private Limited, 2013