Scribd
Upload
2 views22 pages

Nearest Neighbour Based Classifiers - Variants

The document discusses various nearest neighbour classifiers, primarily focusing on the k-Nearest Neighbour (kNN) algorithm, which classifies data based on the similarity to existing cases without making assumptions about the data. It also introduces the Modified k-Nearest Neighbour (MkNN) algorithm, which weights the nearest neighbours by their distance, and the Fuzzy kNN algorithm, which incorporates fuzzy set theory for membership values. Additionally, it mentions r-Near Neighbours, which considers all samples within a specified distance from the test sample.

Uploaded by

phicsp1234
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
2 views22 pages

Nearest Neighbour Based Classifiers - Variants

The document discusses various nearest neighbour classifiers, primarily focusing on the k-Nearest Neighbour (kNN) algorithm, which classifies data based on the similarity to existing cases without making assumptions about the data. It also introduces the Modified k-Nearest Neighbour (MkNN) algorithm, which weights the nearest neighbours by their distance, and the Fuzzy kNN algorithm, which incorporates fuzzy set theory for membership values. Additionally, it mentions r-Near Neighbours, which considers all samples within a specified distance from the test sample.

Uploaded by

phicsp1234
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 22

Nearest Neighbour Based

Classifiers
Nearest neighbour classifiers
k-Nearest Neighbour (kNN) classification algorithm

○ K-Nearest Neighbour is one of the simplest Machine Learning algorithms based on


Supervised Learning technique.
○ K-NN algorithm assumes the similarity between the new case/data and available cases
and put the new case into the category that is most similar to the available categories.
○ K-NN algorithm stores all the available data and classifies a new data point based on the
similarity.
○ K-NN is a non-parametric algorithm, which means it does not make any assumption on
underlying data.
○ It is also called a lazy learner algorithm because it does not learn from the training set
immediately instead it stores the dataset and at the time of classification, it performs an
action on the dataset.
○ KNN algorithm at the training phase just stores the dataset and when it gets new data,
then it classifies that data into a category that is much similar to the new data.
Example:
● The value chosen for k is crucial. With the right value of k, the classification accuracy will
be better than that got by using the nearest neighbour algorithm.
Modified k-Nearest Neighbour (MkNN) Algorithm

● This algorithm is similar to the kNN algorithm, inasmuch as it takes the k nearest
neighbours into consideration.
● The only difference is that these k nearest neighbours are weighted according to
their distance from the test point. It is also called the distance-weighted
k-nearest neighbour algorithm.
Each of the neighbours is associated with the weight w which is defined as

where j = 1, .., k. The value of wj varies from a maximum of 1 for the nearest
neighbour down to a minimum of zero for the most distant.
Note: d1 is the distance of test pattern P to the most nearest neighbour.
Similarly, dk is the dist. Of P to the most farthest neighbout.
● Having computed the weights wj, theMkNN algorithm assigns the test
pattern P to that class for which the weights of the representatives among
the k nearest neighbours sums to the greatest value.
● Instead of using the simple majority rule, it can be observed that MkNN
employs a weighted majority rule. This would mean that outlier patterns have
lesser effect on classification.
Fuzzy kNN Algorithm

● In this algorithm, the concept of fuzzy sets, whose elements have a degree of
membership, is used. In classical set theory, an element either belongs or does
not belong to a set.
● In fuzzy sets, the elements of the set have a membership function attached to
them which is in the real unit interval [0, 1].
● The membership value of each pattern to each class depends on the class of its k
neighbours.
r-Near Neighbours

• Instead of looking at k nearest


neighbours to the test sample P, we look
at all samples falling within a distance r
from P.
The algorithm is as follows.
Use of the Nearest Neighbour Algorithm for Transaction Databases

You might also like