Scribd
Upload
37 views8 pages

K - Nearest Neigbors Hyperparameter: and It's

K-Nearest Neighbors (KNN) is a supervised machine learning algorithm used for classification and regression. It works by finding the k closest training examples in feature space and predicting the label based on a majority vote of its neighbors. Key hyperparameters for KNN include the number of neighbors k, distance metric, weights, and algorithm used. While simple and versatile, KNN can be computationally intensive and sensitive to irrelevant features and data scaling issues.

Uploaded by

mourad.zaied
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
37 views8 pages

K - Nearest Neigbors Hyperparameter: and It's

K-Nearest Neighbors (KNN) is a supervised machine learning algorithm used for classification and regression. It works by finding the k closest training examples in feature space and predicting the label based on a majority vote of its neighbors. Key hyperparameters for KNN include the number of neighbors k, distance metric, weights, and algorithm used. While simple and versatile, KNN can be computationally intensive and sensitive to irrelevant features and data scaling issues.

Uploaded by

mourad.zaied
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

K - Nearest Neigbors

and it’s Hyperparameter

Category B

Category A

New Data Point


What is KNN?
The k-Nearest Neighbors (kNN) algorithm is a
supervised machine learning algorithm used
for classification and regression tasks. It
works by finding a predefined number of
training samples closest in distance to a new
point and predicting the label based on their
labels.
Why to use KNN?
Simplicity: It's an intuitive and easy-to-
implement algorithm.

No Training Phase: kNN is a lazy learner,


which means it doesn't require a separate
training phase and starts working at the
time of prediction.

Adaptive: Can quickly adapt to changes as


it does not require retraining with the
addition of new data.

Useful for non-linear data: Can capture


complex decision boundaries.

Versatile: Can be used for both


classification and regression tasks.
Advantages of KNN

No Assumption: Does not make any


assumptions about the functional form of
the transformation, which makes it
suitable for any type of data.

Instance-Based Learning: Can adapt


easily to changes.

Multiclass Classification: Naturally


supports multi-class classification.
Disdvantages of KNN

Computationally Intensive: Requires the


storage of the entire dataset. Finding the
nearest neighbors can be computationally
expensive.

Sensitive to Irrelevant Features: Since it


relies on distance computation, irrelevant
features can impact performance.

Sensitive to the Scale: Features with


higher scales can dominate the distance
metric. Feature scaling is crucial.

Degrades with High Dimensionality: Due


to the curse of dimensionality, distance-
based methods like kNN can perform
poorly.
Hyperparameters in KNN

Number of Neighbors (k)

Distance Metric (e.g., Euclidean,


Manhattan, Minkowski)

Weights (Uniform, Distance, or Custom)

Algorithm (Brute-force, KD-Tree, Ball Tree)

Leaf Size (For tree-based algorithms)

P (Power parameter for Minkowski


distance)

Metric Params (Additional arguments for


metric function)
Python Code to Implement KNN

You might also like