Scribd
Upload
30 views8 pages

Classification - KNN

The document discusses the K-Nearest Neighbors algorithm for classification. It explains the intuition behind KNN using 1-nearest neighbor and 5-nearest neighbor examples. It then outlines the KNN algorithm steps of selecting K, calculating distances to neighbors, and assigning the data point to the class with the most neighbors. It concludes that KNN is a simple algorithm but can be affected by noise and requires all computation at prediction time.

Uploaded by

Diễm Quỳnh Trần
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
30 views8 pages

Classification - KNN

The document discusses the K-Nearest Neighbors algorithm for classification. It explains the intuition behind KNN using 1-nearest neighbor and 5-nearest neighbor examples. It then outlines the KNN algorithm steps of selecting K, calculating distances to neighbors, and assigning the data point to the class with the most neighbors. It concludes that KNN is a simple algorithm but can be affected by noise and requires all computation at prediction time.

Uploaded by

Diễm Quỳnh Trần
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

K-Nearest Neighbors

2. K-Nearest Neighbor – Intuition (KNN)


Problem
Given two classes A, B and a new data point x. Which’ class the point x belong to?

Manh-Hung Ha ISVNU
2. K-Nearest Neighbor – Intuition (KNN)
Problem
Given two classes A, B and a new data point x. Which’ class the point x belong to?

KNN? 1- Nearest Neighbor


• Calculate distance from point x to all the points in A and B
• Find smallest distance from point x to a point in class A and B
d3 • Check the nearest point belong to class A or B
d1 • The point x is belong to same class with the nearest point
d2
d1 < d2 < d3 …< dm
d1 = min{d1, d2, … dm}

Point “1” belong to class B => Point x belong to class B

Manh-Hung Ha ISVNU
2. K-Nearest Neighbor – Intuition (KNN)
Problem
Given two classes A, B and a new data point x. Which’ class the point x belong to?

1-NN Distance??

d3
d1
d2

Manh-Hung Ha ISVNU
2. K-Nearest Neighbor – Intuition (KNN)
Problem
Given two classes A, B and a new data point x. Which’ class the point x belong to?

5-NN

• Totally 5 neighbors
• 2 neighbors in class B
• 3 neighbors in class A
• New data point x belong to class A

How about KNN?

Manh-Hung Ha ISVNU
3. KNN algorithm
Algorithm
Given multiples classes and a new data point x. Which’s class the point x belong to?
• Select the number K of the neighbors
• Calculate the distance of K number of neighbors
• Take the K nearest neighbors as per the calculated distance.
• Among these K neighbors, count the number of the data points in each category.
• Assign the new data points to that category for which the number of the neighbor is maximum.

Trial test: K = 1, 2, … N
How to find K?
Find K with the best accuracy

Manh-Hung Ha ISVNU
3. KNN algorithm
Algorithm

• Select K (1, 2, …N)


• Calculate distance from x to all data points
• Sort K to take K nearest neighbors
• Count number of points belong to classes
• Point x is belong to the class having max
neighbor points
n
di =

Sort: d1 < d2 < d3 …< dm


Take: {d1, d2, … dk}
Count

Manh-Hung Ha ISVNU
4. Conclusions

• KNN is a simple algorithm


• Complexity ~ numbers of data & number of features
• All features are equal importance
• Easy affected by noise data
• No parameters
• No training time, all calculation in the test time (prediction step)
• Could be used in classification and regression

Manh-Hung Ha ISVNU

You might also like