Scribd
Upload
4 views18 pages

23 Supervised KNN

The K-Nearest Neighbor (K-NN) algorithm is a simple supervised learning technique that classifies new data points based on their similarity to existing data. It operates by storing all available data and using Euclidean distance to determine the closest neighbors, ultimately assigning a category based on majority voting among those neighbors. While K-NN is easy to implement and effective with large datasets, it requires careful selection of the value of K and can be computationally intensive.

Uploaded by

jovita
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
4 views18 pages

23 Supervised KNN

The K-Nearest Neighbor (K-NN) algorithm is a simple supervised learning technique that classifies new data points based on their similarity to existing data. It operates by storing all available data and using Euclidean distance to determine the closest neighbors, ultimately assigning a category based on majority voting among those neighbors. While K-NN is easy to implement and effective with large datasets, it requires careful selection of the value of K and can be computationally intensive.

Uploaded by

jovita
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

K-NEAREST

NEIGHBOR
INTRODUCTION
INTRODUCTION

 K-Nearest Neighbor is one of the simplest Machine Learning algorithms based on Supervised Learning
technique.
 K-NN algorithm assumes the similarity between the new case/data and available cases and put the new case
into the category that is most similar to the available categories.
 K-NN algorithm stores all the available data and classifies a new data point based on the similarity. This
means when new data appears then it can be easily classified into a well suite category by using K- NN
algorithm.
 K-NN is a non-parametric algorithm, which means it does not make any assumption on underlying data.
INTRODUCTION

 It is also called a lazy learner algorithm because it does not learn from the training set immediately instead it
stores the dataset and at the time of classification, it performs an action on the dataset.
 KNN algorithm at the training phase just stores the dataset and when it gets new data, then it classifies that
data into a category that is much similar to the new data.
 Example:
WHY DO WE NEED A K-NN ALGORITHM?

 Suppose there are two categories, i.e., Category A and Category B, and we have a new data point x1, so this
data point will lie in which of these categories.
 To solve this type of problem, we need a K-NN algorithm. With the help of K-NN, we can easily identify the
category or class of a particular dataset.
HOW K-NN WORK
HOW K-NN WORK

The K-NN working can be explained on the basis of the below algorithm:

1. Select the number K of the neighbors


2. Calculate the Euclidean distance of K number of neighbors
3. Take the K nearest neighbors as per the calculated Euclidean distance.
4. Among these k neighbors, count the number of the data points in each category.
5. Assign the new data points to that category for which the number of the neighbor is maximum.
6. Our model is ready.
HOW K-NN WORK
Suppose we have a new data point and we need to put it in the required category.

Firstly, we will choose the number of neighbors, so we will


choose the k=5.
HOW K-NN WORK
Next, we will calculate the Euclidean distance between the data points. The Euclidean distance is the distance
between two points, which we have already studied in geometry.
HOW K-NN WORK
By calculating the Euclidean distance we got the nearest neighbors, as three nearest neighbors in category A and
two nearest neighbors in category B.

As we can see the 3 nearest neighbors are from category A,


hence this new data point must belong to category A.
HOW TO SELECT THE VALUE OF K IN THE K-NN ALGORITHM?

Some points to remember while selecting the value of K in the K-NN algorithm:
 There is no particular way to determine the best value for "K", so we need to try some values to find the best
out of them. The most preferred value for K is 5.
 A very low value for K such as K=1 or K=2, can be noisy and lead to the effects of outliers in the model.
 Large values for K are good, but it may find some difficulties.
HOW TO SELECT THE VALUE OF K IN THE K-NN ALGORITHM?
Let see the effect of value “K” on the class boundaries. The following are the different boundaries separating the two
classes with different values of K.

If you watch carefully, you can see that the boundary becomes smoother with increasing value of K. With K
increasing to infinity it finally becomes all blue or all red depending on the total majority.
HOW TO SELECT THE VALUE OF K IN THE K-NN ALGORITHM?
The training error rate and the validation error rate are two parameters we need to access different K-value.
Following is the curve for the training error rate with a varying value of K :

As you can see, the error rate at K=1 is always zero for the training sample. This is because the closest point to any
training data point is itself.Hence the prediction is always accurate with K=1.
HOW TO SELECT THE VALUE OF K IN THE K-NN ALGORITHM?
If validation error curve would have been similar, our choice of K would have been 1. Following is the validation error
curve with varying value of K:

This makes the story more clear. At K=1, we were


overfitting the boundaries. Hence, error rate initially
decreases and reaches a minima. After the minima
point, it then increase with increasing K. To get the
optimal value of K, you can segregate the training and
validation from the initial dataset. Now plot the
validation error curve to get the optimal value of K.
This value of K should be used for all predictions.
ADVANTAGES & DISADVANTAGES
ADVANTAGES OF KNN ALGORITHM

 It is simple to implement.
 It is robust to the noisy training data
 It can be more effective if the training data is large.
DISADVANTAGES OF KNN ALGORITHM

 Always needs to determine the value of K which may be complex some time.
 The computation cost is high because of calculating the distance between the data points for all the training
samples.
REFFERENCES

 https://www.javatpoint.com/k-nearest-neighbor-algorithm-for-machine-learning
 https://www.analyticsvidhya.com/blog/2018/03/introduction-k-neighbours-algorithm-clustering/
 https://www.codecademy.com/learn/introduction-to-supervised-learning-skill-path/modules/k-nearest-neighbors-
skill-path/cheatsheet

You might also like