Scribd
Upload
255 views2 pages

K-Nearest Neighbours Algorithm: KNN-Visualization

K-nearest neighbours algorithm is a simple machine learning algorithm that can be easily understood and visualized. It works by finding the K closest training examples in feature space to a new data point and predicting the label of the new point based on the majority label of its K nearest neighbors. The time complexity is O(n) where n is the size of the training set. Distance is calculated using the Euclidean distance formula. The value of K and handling of unbalanced classes can affect accuracy. KNN performs well in high dimensions if dimensionality reduction is used first.

Uploaded by

Onkar kunjir
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
255 views2 pages

K-Nearest Neighbours Algorithm: KNN-Visualization

K-nearest neighbours algorithm is a simple machine learning algorithm that can be easily understood and visualized. It works by finding the K closest training examples in feature space to a new data point and predicting the label of the new point based on the majority label of its K nearest neighbors. The time complexity is O(n) where n is the size of the training set. Distance is calculated using the Euclidean distance formula. The value of K and handling of unbalanced classes can affect accuracy. KNN performs well in high dimensions if dimensionality reduction is used first.

Uploaded by

Onkar kunjir
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

K-nearest neighbours algorithm​ is one of the simplest machine learning algorithm and an

great point to start is you are new to machine learning. As not only KNN (K-nearest neighbours)
algorithm is simple to understand and implement but also its very easy to visualise how to the
classification is performed in the algorithm.

In this post you will come to know about KNN algorithm and it's implementation in JavaScript
and for those impatient readers who directly wants to view the source code here is link to source
code of ​KNN-Visualization​ , program uses P5.js library to preform graphical operation and run
the html file in local host.

Now, lets get started with KNN algorithm is instance based learning or lazy learning type
algorithm. KNN predicts the label of point by comparing distance of that point from K data points
in training set. Time complexity for this algorithm is O(n) where n is size of training set.
Distance between to points is calculated using ​euclidean distance formula​.

for 2 dimensional feature space d = sqrt ( pow(x2 - x1 , 2) + pow(y2 - y1 , 2) ) .


After calculating distance of unknown point from each point in training set K points with lowest
distance are selected and major class label is assigned to unknown point and that's it, this is
KNN algorithm.
Advantages :

● Faster to implement
● Can be easily implemented and visualize

Disadvantages :

● Accuracy will be easily affected if the number sample point of one class are very large in
number than other
● The choice of value of K is also very important

The problem with unbalanced number of sample point of classes can be avoided by multiplying
distance with suitable weights proportional to number of sample points and optimal value of K
can be obtained using ​hyperparameter optimization​.
KNN can perform very well for dataset of higher dimension if dimensional reduction is performed
using algorithms such as PCA component analysis.

Now, talking about program that I have implemented to classify iris flower species using KNN
is and web app which shows how the classification is performed based on k-nearest sample
points which are compared to predict its class. If you move cursor over the graph you will be
able to see the lines between new sample point and its k nearest sample points which decides
its label , you can also change the value of K in order to understand how K value affect the
prediction.
The purpose of selecting lines to help visualize the decision instead of region like as follows

is that the region don't exactly gives idea how the prediction was made. So that was all about
KNN algorithm try to implement the algorithm on your own. Happy coding.

You might also like