Scribd
Upload
8 views21 pages

Lecture Slides#7

The document discusses K-Nearest Neighbors (KNN) as a supervised machine learning algorithm used for classification based on the proximity of data points in the training set. It explains how KNN works, the importance of choosing the right value of K, and when to use KNN effectively. Additionally, it contrasts eager and lazy learners in the context of machine learning methodologies.

Uploaded by

kadir.56720
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
8 views21 pages

Lecture Slides#7

The document discusses K-Nearest Neighbors (KNN) as a supervised machine learning algorithm used for classification based on the proximity of data points in the training set. It explains how KNN works, the importance of choosing the right value of K, and when to use KNN effectively. Additionally, it contrasts eager and lazy learners in the context of machine learning methodologies.

Uploaded by

kadir.56720
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 21

MAT1008 MATHEMATICAL DATA

ANALYSIS
Week #8
Machine Learning

Supervised Learning Unsupervised Learning

K-means Clustering
Regression Classification

Linear Regression Logistic Regression

Naive Bayes

K-Nearest Neighbours
Decision Trees
Support Vector Machines
Artificial Neural Networks
K-Nearest Neighbours
K-Nearest Neighbors (KNN) Classifier
K-Nearest Neighbors (KNN) Classifier
K-Nearest Neighbors (KNN) Classifier

• An intuitive way to decide how to classify an unlabelled test item is to


look at the training data points nearby, and make the classification
according to the classes of those nearby labelled data points. This
intuition is formalised in a classification approach called K-nearest
neighbour (k-NN) classification.
K-Nearest Neighbors (KNN) Classifier
K-Nearest Neighbors (KNN) Classifier

• KNN is one of the simplest Supervised ML algorithms mostly used for


classifying a data point based on how its neighbors are classified

• Classifies new data points based on a similarity measure.


How KNN Works
How KNN works?
K= 5
Pr (CATS|X) = 4/5
Pr(DOGS|X) = 1/5
Hence, KNN
predicts that X
belongs to class
CATS.
How KNN works?
For each test example:
• Compute the distance between test example and each training example
• Select the set of the k nearest training examples to test example
• Decide the class by the majority voting
How to calculate the distance?
• You find the distance between points using distance measures such as
Euclidean distance, Hamming distance, Manhattan distance or
Minkowski distance.
• Euclidean Distance:
How to choose K ?
How to choose K ?
• Choosing the right value of K is a process called Parameter Tuning
and it is important for better accuracy.
• Research has shown that no optimal number of neighbors suits all
kind of data sets. Each dataset has it's own requirements.
• Data scientists choose as an odd number if the number of classes is
even.
• You can also check by generating the model on different values of k
and check their performance.
• K may be Sqrt(n), where n is the number of data points.
When to use KNN ?

• KNN performs better with a lower number of features than a large


number of features. When the number of features increases than it
requires more data.
• Data set is small.

• Data is labeled.

• Data is noise free.


Eager vs Lazy Learners

• Eager learners mean when given training points will construct a


generalized model before performing prediction on given new points
to classify.

• We can think of such learners as being ready, active and eager to


classify unobserved data points.
Eager vs Lazy Learners
• Lazy Learning means there is no need for learning or training of the
model and all of the data points used at the time of prediction.
• Lazy learners wait until the last minute before classifying any data point. Lazy
learner stores only the training dataset and waits until classification needs to
perform. Only when it sees the test tuple, it classifies the tuple based on its
similarity to the stored training tuples.
• Unlike eager learning methods, lazy learners do less work in the training
phase and more work in the testing phase to make a classification.
• Lazy learners are also known as Instance-Based Learners because lazy
learners store the training instances, and all learning is based on instances.
In-Class Work # 3

• Based on the given data, classify a person with 57 kg and 170 cm


height as Underweight or Normal using KNN algorithm.
In-Class Work # 3
In-Class Work # 3
References
• An Introduction to Statistical Learning – with Applications in R,” G.
James, D. Witten, T. Hastie, R. Tibshirani. 7th Edition, Springer, New
York, 2013
• KNN Algorithm - How KNN Algorithm Works With Example |
Simplilearn

You might also like