Scribd
Upload
8 views

K_NN classification

Idk

Uploaded by

kavitadikshanayal26
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
8 views

K_NN classification

Idk

Uploaded by

kavitadikshanayal26
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

K-Nearest Neighbors (KNN) Algorithm

K-Nearest Neighbors (KNN) is a simple, intuitive machine learning algorithm used for
both classification and regression tasks. The core idea behind KNN is that data points
that are close to each other in feature space are likely to belong to the same class (for
classification) or have similar values (for regression).

How KNN Works (Step-by-Step):

1. Choose the number of neighbors (K): You first select the number of nearest
neighbors to consider for making a decision. For example, if K = 3, we consider
the 3 closest points to the new point.
2. Calculate the distance: For a new data point, calculate the distance from it to all
other points in the training dataset. A common distance metric is Euclidean
distance.
3. Sort the distances: Once you calculate all the distances, sort them in ascending
order to find the K nearest points (neighbors).
4. Make a prediction:
o For classification: Assign the most common class (the majority class)
among the K nearest neighbors to the new point.
o For regression: Take the average(or mean) of the values of the K nearest
neighbors and assign it to the new point.
Example 2 : Classification of Fruits

Let's say we have a dataset of fruits based on two features: weight and color intensity.
We need to classify a new fruit as either an apple or a banana.
| Fruit | Weight (grams) | Color Intensity (scale 1-10) | Class |
| Apple | 150 |8 | Apple |
| Apple | 160 |7 | Apple |
| Banana | 120 |4 | Banana |
| Banana | 130 |5 | Banana |
| Apple | 140 |8 | Apple |
| Banana | 125 |6 | Banana |

Step-by-step Process

1. Choose k = 3: We'll consider the 3 nearest neighbors.


2. New data point: A new fruit has weight 135 grams and color intensity 7. We want to
classify it as either an apple or a banana.
3. Calculate distances: We calculate the Euclidean distance between the new point and
all other points in the dataset:
distance formula: ( sqrt((x_1 - x_2)^2 + (y_1 - y_2)^2) )
- Distance to first apple (150, 8): ( sqrt ((135-150)^2 + (7-8)^2 ) = sqrt(225 + 1 ) = sqrt(226) =15.03
- Distance to second apple (160, 7): ( sqrt ((135-160)^2 + (7-7)^2) = sqrt (625 ) = 25
- Distance to first banana (120, 4): ( sqrt ((135-120)^2 + (7-4)^2) = sqrt (225 + 9) = sqrt (234 ) =15.30
- Distance to second banana (130, 5): ( sqrt ((135-130)^2 + (7-5)^2 ) = sqrt (25 + 4 ) = sqrt (29 )=5.39
- Distance to third apple (140, 8): (sqrt ((135-140)^2 + (7-8)^2 ) = sqrt (25 + 1 ) = sqrt (26 ) = 5.10
- Distance to third banana (125, 6): ( sqrt ((135-125)^2 + (7-6)^2 ) =sqrt (100 +1)= sqrt (101) =10.05

4. Sort by distance: The 3 nearest points (smallest distances) are:


- Second apple (140, 8), distance ≈ 5.10
- Second banana (130, 5), distance ≈ 5.39
- First apple (150, 8), distance ≈ 15.03

5. Classify: The majority of the 3 nearest neighbors are apples, so the new fruit is
classified as an apple.

You might also like