Scribd
Upload
13 views15 pages

Machine Learning 05

The document discusses non-linear classifiers, specifically the XOR problem, which cannot be separated by a single hyperplane. It introduces the two-layer perceptron architecture that transforms non-linearly separable problems into linearly separable ones through a mapping process. Additionally, it covers the capabilities of three-layer perceptrons to classify vectors into any union of polyhedral regions, enhancing classification flexibility.

Uploaded by

233046
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
13 views15 pages

Machine Learning 05

The document discusses non-linear classifiers, specifically the XOR problem, which cannot be separated by a single hyperplane. It introduces the two-layer perceptron architecture that transforms non-linearly separable problems into linearly separable ones through a mapping process. Additionally, it covers the capabilities of three-layer perceptrons to classify vectors into any union of polyhedral regions, enhancing classification flexibility.

Uploaded by

233046
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 15

Non Linear Classifiers

 The XOR problem


x1 x2 XOR Class
0 0 0 B
0 1 1 A
1 0 1 A
1 1 0 B

1
 There is no single line (hyperplane) that
separates class A from class B. On the
contrary, AND and OR operations are linearly
separable problems

2
 The Two-Layer Perceptron

 For the XOR problem, draw two, instead, of one


lines

3
 Then class B is located outside the shaded area
and class A inside. This is a two-phase design.
• Phase 1: Draw two lines (hyperplanes)

g1 ( x)  g 2 ( x) 0
Each of them is realized by a perceptron. The
outputs of the perceptrons will be

0
yi  f ( g i ( x))  i 1, 2
1
depending on the position of x.

• Phase 2: Find the position of x w.r.t. both lines,


based on the values of y1, y2.
4
Perceptron

5
1st phase 2nd
x1 x2 y1 y2 phase

0 0 0(-) 0(-) B(0)


0 1 1(+) 0(-) A(1)
1 0 1(+) 0(-) A(1)
• 1 1 The
Equivalently: 1(+) 1(+)
computations B(0)of the first
x  y [ y1 , y2 ]T
phase perform a mapping

6
 Computations of the first phase perform a
mapping that transforms the nonlinearly
separable problem to a linearly separable
one.

 The architecture

7
• This is known as the two layer
perceptron with one hidden and one
output layer. The activation functions
are
0
f (.) 
1

• The neurons (nodes) of the figure realize


the following lines (hyperplanes)
1
g1 ( x)  x1  x2  0
2
3
g 2 ( x)  x1  x2  0
2
1
g ( y )  y1  2 y2  0
2 8
 Classification capabilities of the two-layer perceptron

 The mapping performed by the first layer neurons is


onto the vertices of the unit side square, e.g.,
(0, 0), (0, 1), (1, 0), (1, 1).

 The more general case,

x  Rl
x  y [ y1 ,... y p ]T , yi  0, 1 i 1, 2,... p 9
performs a mapping of a vector
onto the vertices of the unit side Hp hypercube

 The mapping is achieved with p neurons each


realizing a hyperplane. The output of each of
these neurons is 0 or 1 depending on the
relative position of x w.r.t. the hyperplane.

10
 Intersections of these hyperplanes form regions
in the l-dimensional space. Each region
corresponds to a vertex of the Hp unit
hypercube.

11
For example, the 001 vertex corresponds to
the region which is located

to the (-) side of g1 (x)=0


to the (-) side of g2 (x)=0
to the (+) side of g3 (x)=0

12
 The output neuron realizes a hyperplane in the
transformedy space, that separates some of the
vertices from the others. Thus, the two layer
perceptron has the capability to classify vectors
into classes that consist of unions of
polyhedral regions. But NOT ANY union. It
depends on the relative position of the
corresponding vertices.

13
 Three layer-perceptrons

 The architecture

 This is capable to classify vectors into classes


consisting of ANY union of polyhedral regions.

 The idea is similar to the XOR problem. It


realizes more than one planesp in the
yR 14
space.
 The reasoning
• For each vertex, corresponding to class, say
A, construct a hyperplane which leaves THIS
vertex on one side (+) and ALL the others to
the other side (-).
• The output neuron realizes an OR gate

 Overall:

The first layer of the network forms the


hyperplanes, the second layer forms the
regions and the output neuron forms the
classes.

 Designing Multilayer Perceptrons


 One direction is to adopt the above rationale
and develop a structure that classifies correctly
all the training patterns.
 The other direction is to choose a structure and
compute the synaptic weights to optimize a cost 15
function.

You might also like