Scribd
Upload
147 views3 pages

Lazy Learning (Or Learning From Your Neighbors)

Lazy learning is an approach where a classifier is not constructed until new instances need classification. It involves storing training data and waiting until new instances need to be classified. Typical lazy learning approaches include k-nearest neighbors and case-based reasoning. The k-nearest neighbors method classifies new instances based on a vote of its k closest neighbors in the training set, where k is a positive integer. It requires a feature space, distance metric, and value of k.

Uploaded by

Diyar T Alzuhairi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
147 views3 pages

Lazy Learning (Or Learning From Your Neighbors)

Lazy learning is an approach where a classifier is not constructed until new instances need classification. It involves storing training data and waiting until new instances need to be classified. Typical lazy learning approaches include k-nearest neighbors and case-based reasoning. The k-nearest neighbors method classifies new instances based on a vote of its k closest neighbors in the training set, where k is a positive integer. It requires a feature space, distance metric, and value of k.

Uploaded by

Diyar T Alzuhairi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

Lazy Learning

(Or Learning from Your Neighbors)


Lazy Learning vs. Eager Learning
 Lazy learning vs. Eager learning
– Eager learning
 Given a set of training set, constructs a classification model before receiving new
(e.g., test) data to classify
 e.g. decision tree induction, Bayesian classification, rule-based classification
– Lazy learning
 Simply stores training data (or only minor processing) and waits until it is given a new
instance
 Lazy learners take less time in training but more time in predicting
 e.g., k-nearest-neighbor classifiers, case-based reasoning classifiers

Typical approaches of lazy learning:

–k-nearest neighbor approach

 Instances represented as points in a Euclidean space.

–Case-based reasoning

 Uses symbolic representations and knowledge-based inference Locally weighted


regression

k-Nearest-Neighbor Method

–first described in the early 1950s

–It has since been widely used in the area of pattern recognition.

–The training instances are described by n attributes.

–Each instance represents a point in an n-dimensional space.

–A k-nearest-neighbor classifier searches the pattern space for the k training instances that are
closest to the unknown instance.

 Requires 3 things:
o Feature Space (Training Data)
o Distance metric
 to compute distance between records
o The value of k
 the number of nearest neighbors to retrieve from which to get
majority class
 To classify an unknown record:
o Compute distance to other training records
o Identify k nearest neighbors
o Use class labels of nearest neighbors to determine the class label of unknown
record

 k = 1:

– Belongs to square class

 k = 3:

? – Belongs to triangle class

 k = 7:

– Belongs to square class

 Choosing the value of k:

o If k is too small, sensitive to noise points

o If k is too large, neighborhood may include points from other classes

o Choose an odd value for k, to eliminate ties

 Common Distance Metrics:

o Euclidean distance (continuos distribution)

Examples
Name Acid Durability Strength class
Type-1 7 7 Bad
Type-2 7 4 Bad
Type-3 3 4 Good
Type-4 1 4 Good
Test – data -----> acid durability = 3 , and strength = 7 , class= ?
 Calculated using distance measure

Name Acid durability Strength Class Distance


Type-1 7 7 Bad Sqrt((7-3)2+(7-7)2)=4
Type-2 7 4 Bad 5
Type-3 3 4 Good 3
Type-4 1 4 Good 3.6

Name Acid durability Strength Class Distance Rank


Type-1 7 7 Bad 4 3
Type-2 7 4 Bad 5 4
Type-3 3 4 Good 3 1
Type-4 1 4 Good 3.6 2
K =1
Name Acid durability Strength Class Distance Rank
Type-1 7 7 Bad 4 3
Type-2 7 4 Bad 5 4
Type-3 3 4 Good 3 1
Type-4 1 4 Good 3.6 2
Based on immediate neighbor, Good
K=2
Name Acid durability Strength Class Distance Rank
Type-1 7 7 Bad 4 3
Type-2 7 4 Bad 5 4
Type-3 3 4 Good 3 1
Type-4 1 4 Good 3.6 2
Based on two neighbor, Good
K=3
Name Acid durability Strength Class Distance Rank
Type-1 7 7 Bad 4 3
Type-2 7 4 Bad 5 4
Type-3 3 4 Good 3 1
Type-4 1 4 Good 3.6 2
Based on three neighbor, 2 Good and 1 Bad, majority---> Good

You might also like