Scribd
Upload
15 views84 pages

Unit 3 (B) NGP

Instance based learning refers to techniques that produce predictions based on similarity to nearest neighbors in training data. It is also known as lazy learning since processing is delayed until a new instance requires classification. Common algorithms include KNN, SOM, LVQ and LWL.

Uploaded by

animehv5500
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
15 views84 pages

Unit 3 (B) NGP

Instance based learning refers to techniques that produce predictions based on similarity to nearest neighbors in training data. It is also known as lazy learning since processing is delayed until a new instance requires classification. Common algorithms include KNN, SOM, LVQ and LWL.

Uploaded by

animehv5500
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 84

Instance Based Learning

• K-nearest neighbor learning


• Locally weighted regression
• Radial basis function (RBF) networks
• Case based learning
Step 2: Find Rank
Most of the voting goes to False, so new instance will be classified as False.
(1.2+2.1+1.8)/3= 1.7
K Means
Clustering
Converged and Finally data points,
A1,B1,C2 belongs to cluster-1
A3,B2,B3 belongs to cluster-2
A2,C1 belongs to cluster-3
when there exists a non-linear relationship between X and Y. In such cases, locally weighted linear
regression is used.
• Locally weighted linear regression is a non-parametric algorithm, that
is, the model does not learn a fixed set of parameters as is done in
ordinary linear regression. Rather parameters are computed
individually for each query point x . While computing, a higher
“preference” is given to the points in the training set lying in the
vicinity of x than the points lying far away from x .
Locally Weighted Regression
• Locally weighted linear regression is a supervised learning algorithm.
• It is a non-parametric algorithm.
• There exists No training phase. All the work is done during the testing
phase/while making predictions.
• Locally weighted regression methods are a generalization of k-Nearest
Neighbour.
• In Locally weighted regression an explicit local approximation is
constructed from the target function for each query instance.
• The local approximation is based on the target function of the form
like constant, linear, or quadratic functions localized kernel functions.
How can you relate KNN Algorithm to the
Bias-Variance tradeoff?
Problem with having too small K: lead to a large variance in the
predictions.

Problem with having too large K:


• The larger the value of K, the higher is the accuracy.
• choosing k to a large value may lead to a model with a large
bias(error).
How to choose the optimal value of K in the
KNN Algorithm?
• No method is the rule of thumb for finding the K value in the
KNN algorithm. Choosing the right value of K is done through a
process known as Hyperparameter Tuning. You have to play
around with different values to choose which value of K should
be optimal for my problem statement. The optimum value of K
for KNN is highly dependent on the data itself.
• Cross-Validation Method: We should also take the help of
cross-validation to find out the optimal value of K in KNN. Start
with the minimum value of k i.e, K=1, and run cross-validation,
measure the accuracy, and keep repeating till the results become
consistent.
• As the value of K increases, the error usually goes down after
each one-step increase in K, then stabilizes, and then raises
again. Finally, pick the optimum K at the beginning of the stable
zone. This technique is also known as the Elbow Method.
Why is it recommended not to use the KNN
Algorithm for large datasets?
• The Problem in processing the data:
KNN works well with smaller datasets because it is a lazy learner.
It needs to store all the data and then make a decision only at run
time. It includes the computation of distances for a given point
with all other points. So if the dataset is large, there will be a lot of
processing which may adversely impact the performance of the
algorithm.
What is space and time complexity of the
KNN Algorithm?
• Time complexity:
• The distance calculation step requires quadratic time complexity, and
the sorting of the calculated distances requires an O(N log N) time.
Together, we can say that the process is an O(N3 log N) process, which
is a monstrously long process.
• Space complexity:
• Since it stores all the pairwise distances and is sorted in memory on a
machine, memory is also the problem. Usually, local machines will crash,
if we have very large datasets.
RBF (Radial Basis Function) Networks
• Radial Basis Functions (RBF) are real-valued functions that use
supervised machine learning (ML) to perform as a non-linear
classifier. Its value depends on the distance between the input and a
certain fixed point.
• Radial Basis Function (RBF) Networks are a particular type of Artificial
Neural Network used for function approximation problems.
• Since Radial basis functions (RBFs) have only one hidden layer, the
convergence of optimization objective is much faster.
• RBF networks have other applications like function approximation,
interpolation, classification and time series prediction.
Architecture of RBF
• RBF network is an artificial neural network with an input layer, a
hidden layer, and an output layer. The Hidden layer of RBF consists of
hidden neurons, and activation function of these neurons is a
Gaussian function. Hidden layer generates a signal corresponding to
an input vector in the input layer, and corresponding to this signal,
network generates a response.

• https://www.hackerearth.com/blog/developers/radial-basis-function-
network/#:~:text=Architecture%20of%20RBF,neurons%20is%20a%20
Gaussian%20function.
• Filter methods measure the relevance of features by their
correlation with dependent variable while wrapper methods
measure the usefulness of a subset of feature by training a
model on it.
• Filter methods are much faster compared to wrapper methods
as they do not involve training the models. On the other hand,
wrapper methods are computationally very expensive as well.
• Filter methods use statistical methods for evaluation of a subset
of features while wrapper methods use cross validation.

Wrapper method:

Filter method:
Instance-based learning
• Instance-based learning refers to a family of techniques for
classification and regression, which produce a class
label/predication based on the similarity of the query to its
nearest neighbor(s) in the training set.
• Lazy learning refers to machine learning processes in which
generalization of the training data is delayed until a query
is made to the system.
• This type of learning is also known as Instance-based Learning.
Lazy classifiers are very useful when working with large
datasets that have a few attributes.
Instance-based learning
• The systems that learn the training examples by heart and then generalizes
to new instances based on some similarity measure.
• It is called instance-based because it builds the hypotheses from the
training instances.
• It is also known as memory-based learning or lazy-learning because they
delay processing until a new instance must be classified.
• The time complexity of this algorithm depends upon the size of training
data. Each time whenever a new query is encountered, its previously stores
data is examined. And assign to a target function value for the new
instance.
• The worst-case time complexity of this algorithm is O (n), where n is the
number of training instances.
Instance based learning
• Instance based learning is a supervised classification learning
algorithm that performs operation after comparing the current
instances with the previously trained instances, which have been
stored in memory.
• Instance: A single row of data is called an instance. It is an
observation from the domain.
• Feature: A single column of data is called a feature. It is a
component of an observation and is also called an attribute of a
data instance.
Instance based learning
• Advantages:
1.Instead of estimating for the entire instance set, local approximations
can be made to the target function.
2.This algorithm can adapt to new data easily, one which is collected as
we go .
• Disadvantages:
1.Classification costs are high
2.Large amount of memory required to store the data, and each query
involves starting the identification of a local model from scratch.
Some of the instance-based learning algorithms are :

1.K Nearest Neighbor (KNN)


2.Self-Organizing Map (SOM)
3.Learning Vector Quantization (LVQ)
4.Locally Weighted Learning (LWL)
5.Case-Based Reasoning

You might also like