Instance – based Learning
Instance – based Learning
Instance-based learning methods simply store the
training examples
Generalizing beyond these examples is postponed
until a new instance must be classified
Each time a new query instance is encountered, its
relationship to the previously stored examples is
examined in order to assign a target function value
for the new instance
Instance – based Learning
Different Learning Methods
Eager Learning
Explicit description of target function on the whole
training set
Instance-based Learning
Learning=storing all training instances
Classification=assigning target function to a new
instance
Referred to as “Lazy” learning
Instance-based Learning
Its very similar to a
Desktop!!
Instance-based Learning
K-Nearest Neighbor Algorithm
Weighted Regression
Radial Basis Functions
Case-based reasoning
K-Nearest Neighbor
All instances correspond to points in an n-dimensional
Euclidean space
Classification is delayed till a new instance arrives
Classification done by comparing feature vectors of
the different points
Target function may be discrete or real-valued
1-Nearest Neighbor
3-Nearest Neighbor
K-Nearest Neighbor
The k- Nearest Neighbor algorithm for
approximation a discrete-valued target function
The K- Nearest Neighbor algorithm for
approximation a real-valued target function
Example
Exercise
Customer Age Income (K) No. of cards Response
John 35 35 3 Yes
Rachel 22 50 2 No
Ruth 63 200 1 No
Tom 59 170 1 No
Neil 25 40 4 Yes
David 37 50 2 ?
Answer
3-
x1 x2 x3 d Rank Nearest
35 35 3 1
15.16575 Yes
22 50 2 54.6626 3 No
63 200 1 5
209.6902 No
59 170 1 179.95 4 No
25 40 4 47.3392 2 Yes
One leave out cross validation KNN-
Watch
https://www.youtube.com/watch?v=Lvs6G_gdgCM
Distance –Weighted Nearest
Neighbor Algorithm
The refinement to the k-NEAREST NEIGHBOR Algorithm
is to weight the contribution of each of the k neighbors
according to their distance to the query point xq, giving
greater weight to closer neighbors.
For example, in the k-Nearest Neighbor algorithm,
which approximates discrete-valued target functions, we
might weight the vote of each neighbor according to the
inverse square of its distance from xq
Distance-Weighted Nearest Neighbor Algorithm for
approximation a discrete-valued target functions
Distance-Weighted Nearest Neighbor Algorithm for
approximation a Real-valued target functions
Locally Weighted Regression
Locally Weighted Regression
Case-Based Reasoning (CBR)
CBR instances are typically represented using
more rich symbolic descriptions, and the methods
used to retrieve similar instances are
correspondingly more elaborate
CBR has been applied to problems such as
conceptual design of mechanical devices based on
stored library of previous design
Case-Based Reasoning in CADET
Instances represented by rich structural descriptions
Multiple cases retrieved (and combined) to form solution
to new problem
Tight coupling between case retrieval and problem
solving
Bottom line:
Simple matching of cases useful for tasks such as
answering help-desk queries
Area of ongoing research
28
Case-Based Reasoning in CADET
CADET: 75 stored examples of mechanical devices
each training example:
<qualitative function, mechanical structure>
new query: desired function
target value: mechanical structure for this function
Distance metric: match qualitative function descriptions
A stored case : T-junction pipe
Stru cture : Function:
Q 1 ,T 1 T = temperature Q1 +
Q = waterflow
Q3
Q2 +
Q 3 ,T 3
T1 +
T3
T2 +
Q 2 ,T 2
A problem specification : Water faucet
+
Stru cture : Function: Cc Qc +
+ + + Qm
? Ch Qh
-
+
Tc
+
+ Tm
Th +
Neural Network
Based on nature, neural networks are the usual
representation we make of the brain : neurons
interconnected to other neurons which forms a
network.
A simple information transits in a lot of them
before becoming an actual thing, like “move the
hand to pick up this pencil”.
Artificial Neural Network (ANN)
The operation of a complete
neural network is
straightforward
One enter variables as
inputs (for example an
image if the neural
network is supposed to tell
what is on an image), and
after some calculations, an
output is returned
(following the first
example, giving an image
Neural networks can usually be read from left to right. Here, the first
layer is the layer in which inputs are entered. There are 2 internals
layers (called hidden layers) that do some math, and one last layer
that contains all the possible outputs.
What does a neuron do ?
Take all values from connected neurons multiplied by their
respective weight, add them, and apply an activation function.
Then, the neuron is ready to send its new value to other neurons.
Activation Function
Activationfunction usually serves to turn the total value
calculated before to a number between 0 and 1
Radial Basis Function (RBF)
Radial basis function (RBF) networks have a fundamentally different
architecture than most neural network architectures.
Mostneural network architecture consists of many layers and introduces
nonlinearity by repetitively applying nonlinear activation functions.
RBFnetwork on the other hand only consists of an input layer, a single
hidden layer, and an output layer.
Radial Basis
The input layer is not a computation
Function (RBF) layer, it just receives the input data
and feeds it into the special hidden
layer of the RBF network.
The computation that is happened
inside the hidden layer is very
different from most neural networks,
and this is where the power of the
RBF network comes from.
The output layer performs the
prediction task such as classification
or regression.
Ref
https://www.youtube.com/watch?v=pAJSisdFpeI
