12

CHAPTER
Instance-Based Learning
(IBL)
12.1 INTRODUCTION
The instance-based learning is
classification and regression
asupervised learning technique
tasks. The instance-based which is used for
after conparing the current learning
ealled as "Lazy Learning" or instances (examples) with the previous performs operation
"Memory-based
next step is taken only after the Learning". Because, in
instances. It is als0
this technigue the
next instance is arrived.
12.1.1 Types of Instance-based
1. K-nearest Learning Methods
Neighbourhood (K-NN) Algorithm
2. Locally Weighted
3. Radial Basis Regression Algorithm
Function Network
4. Case-based Learning
5. Learning Vector
6. Self Quantization (LVQ)
Organising Map (SOM)
Approaches for Instance Based Learning

1. K-NN
2. Locally Weighted 3. Radial Basis
Regression (LWR) Function (RBF)
4. Case Based
Learning (CBL)
5. Learning Vector
Quantization (LVQ)
6. Self Organising
Map (SOM)
Fig. 12.1. Types of instance-based
learning methods
 EARNING
12.2 K-NEAREST NEIGHBOUR (K-NN) ALGORITHM

The K-Nearest Neighbour algorithm is aa most widely used supervised

algorithm. It can be used for both classification and regression but mostly learning
it is
used
for classification task only.
+ +
Nearest Nearest
neighbour neighbour
+
Query Query Area of
Instance
(X) Query instance Hypothesis
space
(H)
Instance (x)
(X)
(a) 1-nearest neighbour (b) 2-nearest neighbour (c) 3-nearest neighbour
points points points

Fig. 12.2. K-Nearest neighbours data points of query instance (X)

In K-NN algorithm, the each new training example is compared with old training
data for classification. K-NN uses "feature similarity" to predict the values of new data
points. Each new data point is assigned a value based on how closely it matches to
previous data points stored in memory.
12.2.1 K-NN Algorithm Steps
The following are the steps for classification task by using K-NN algorithm.
Step 1. Load the data set.
Step 2. Choose any value of nearest neighbour data points (K). K can be any integer
value (say K= 5).
Step 3. Calculate the Euclidean distance between test data (Query instance, x) and
each row of training data. Note down the value of Eucledean distance in an
ordered table. (Say Table 12.1)

Before K-NN After K-NN

Category B Category B

New data point New data point

Query Instance (x) assigned to
K-NN Category 1
CategoryA Category A +X

Fig. 12.3. Before and after KNN

 241

Step 4. Arrange the Euclidean distance table in

(say K= 5) from this table. (See Table 12.1)ascending order. Choose top Krows
Step 5. Now assign a class to the new instance point query (test
nearest classof rows. Thus, assign the newdata data) based on most
for which the number of neighbour is maximum. point to that category (class)
Step 6. End.

Prample 12.1. Let us consider Fig. 12.4. in which we have a new data point. Our task
is toput it in the required category (class) as class A or class B.

Category B

New Query Instance (x)

Category A

Fig. 12.4. New query instance (Test data)

Solution.
Step 1. Choose number of neighbours (K).
Step 2. Calculate Euclidean distance between new instance and all data points using
following formula. (Fig. 12.5).
Euclidean Distance = (x -X) +(Y, - Y,) ...(12.1)

AB(X, Y)

TAX, Y,)
X
X,

Fig. 12.5. Euclidean distance

Table 12.1. From the Table 12.1, it is

Step 3. Note down Euclidean distance in a neighbour points in category A and
observed that there are say three nearest
(Fig. 12.6).
two pointsnear tocategory B
 242
MACHINE LEARNING,
Table 12.1. Example training data
S.No. X Euclidean distance
1. 10 70 3

2. 20 80 5

3 50 100
:
.

Step 4. As there are more points as neighbours of category A, therefore, we

assign new
data point class at categoryA class. Hence, classification of new data
completed. point

Category B CategoryA:3 neighbors

Category B:2 neighbors
New Data
point
Query Instance (x)
CategoryA
X.

Fig. 12.6. Assigning category (Class)

Example 12.2. The result marks of five students are given in Table 12.2. Now, a te
query is arrived about a student (X) as student (X) marks.
Table 12.2. Training Data (Student's marks)
S.No. Math (X) Science (Y) Result
1. Fail
2. 6 7 Pass
3. 7 Pass
4. 5 5 Fail
5. 8 Pass

Math 6
Test Data
Science 8
Now, our task is to tell whether this new student (X) is pass or fail.
 (NSTANCE-BASED LEARNING (IBL) 243
NG
Solution.

Step 1. Get data and choose (K=3).

CueD 2. Calculate Euclidean distance between new student (X) marks with all students
of Table 12.1.
d= J(x - X,)² +(Y, -Y,)² ...(12.2)
Step 3: Euclidean Distance Calculations:
For point of row 1
W(6-4) + (83) = 5.38
For point of row 2
V6-6) +(8-7) =1
For point of row 3
N(6-7) +(8 8) =1
For point of row 4
V(6-5) +(8-5) =3.16
For point of row 5
N(6-8)+ (8-8) =2
Table 12.3. Euclidean distance (Ascending order)
Row 2 1

Row 3 1
Row 5 2 Increasing order
Row 4 3.16
Row 1 5.38

Step 4. Now pick top 3rows of Euclidean Table 12.3 and check. All
rows 2, 3, 5 belong to category "Pass". So we decide that the new
these points of
student (X)
is a "pass" category student. Hence, classification task is done.
Example 12.3. Python Program Implimentation of K-NN Algorithm (KNN as
an Classifier algorithmn)
"First, start with importing necessary python packages:
1. import numpy as np
2. import matplotlib.pyplot as plt
3. import pandas as pd