Scribd
Upload
100 views12 pages

Xor Gate: Perceptron Can Not Realize An XOR Gate. We Need More Complex Network or Use Different Transfer Functions

Perceptron cannot realize an XOR gate and more complex networks or transfer functions are needed. The decision boundary is the surface where the output equals the threshold, dividing the space into two regions. XOR requires two decision boundaries that cannot be represented by a single hyperplane, making it non-linearly separable.

Uploaded by

Rahul Singh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
100 views12 pages

Xor Gate: Perceptron Can Not Realize An XOR Gate. We Need More Complex Network or Use Different Transfer Functions

Perceptron cannot realize an XOR gate and more complex networks or transfer functions are needed. The decision boundary is the surface where the output equals the threshold, dividing the space into two regions. XOR requires two decision boundaries that cannot be represented by a single hyperplane, making it non-linearly separable.

Uploaded by

Rahul Singh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 12

XOR GATE

Perceptron can not realize an XOR gate. We need more


complex network or use different transfer functions.
The Threshold as a Special Kind of Weight
Decision Boundaries
The decision boundary is the surface at which the output of the unit is
precisely equal to the threshold
On one side of this surface, the output Y will be 0, and on the other side it will be 1.
In two dimension the surface is w1x1 + w2x2 = θ
Which we can write x2 = (-w1/w2)x1+ θ/w2
which is the equation of a line of slope -w1/w2 with intercept θ/w2

x1
w1

1 w1
 slope = 
y w2
2 w2

w2

0 2
x2

W1=1, w2=2,   2
Decision Boundaries for AND and OR

we can see that the gradient of the decision surface


should be negative, and the intercept should be between
0 & 1 for implementing OR logic (& greater than 1 for
AND logic)
Decision Boundaries for XOR

We need two straight lines to separate the different


outputs/decisions

There are two obvious remedies: either change the transfer


function so that it has more than one decision boundary, or
use a more complex network that is able to generate more
complex decision boundaries.
Decision Hyperplanes and Linear Separability
If we have two dimensional inputs, then the weights define a
decision boundary that is a one dimensional straight line in the
two dimensional input space of possible input values.
If we have n dimensional inputs, the weights define a decision
boundary that is an n-1 dimensional hyperplane in the n
dimensional input space:

This hyperplane is clearly still linear and can still only divide the
space into two regions. We still need more complex transfer
functions, or more complex networks, to deal with XOR type
problems.

Problems with input patterns which can be classified using a single


hyperplane are said to be linearly separable. Problems (such as
XOR) which cannot be classified in this way are said to be non-
linearly separable.
Learning and Generalization
Classification: The ability to assign an input observation to a
category
Learning is a relatively permanent change in behaviour brought
about by experience.
Learning The network must learn decision surfaces from a set of
training patterns so that these training patterns are classified
correctly.
Generalization After training, the network must also be able to
generalize, i.e.correctly classify test patterns it has never seen
before.
Usually we want our neural networks to learn well, and also to
generalize well.
Sometimes, the training data may contain errors (e.g. noise in the
experimental determination of the input values, or incorrect
classifications). In this case, learning the training data perfectly
may make the generalization worse. There is an important
tradeoff between learning and generalization that arises quite
generally.
Generalization in Classification
Generalization in Function Approximation
Activation functions
• Transforms neuron’s input into output.
• Features of activation functions:
• A squashing effect is required
• Prevents accelerating growth of activation levels through the
network.

Step function Sign function Sigmoid function Linear function

Y Y Y Y
+1 +1 +1 +1

0 X 0 X 0 X 0 X
-1 -1 -1 -1

1, if X  0  1, if X  0 1
Y step   Y sign   Y sigmoid  Y linear  X
0, if X  0  1, if X  0 1  e X
1

0.8

0.6

0.4 1

0.2 0.5

0
-5 0 5 0

-0.5

• The hard-limiting threshold function -1


-5 0 5

– Corresponds to the biological paradigm


• either fires or not

 1, net  0
f (net )  sgn( net )   bipolar binary
  1, net  0

 1, net  0
f (net )  sgn( net )   unipolar binary
0, net  0
2
f (net )   1 bipolar continuous
1  e ( net)
 1, net  0
f (net )  sgn( net )   bipolar binary
  1, net  0
1
f (net )  ( net)
unipolar continuous
1 e
 1, net  0
f (net )  sgn( net )   unipolar binary
0, net  0

0.9

0.8

0.7

0.6

0.5
a=logsig(n) = 1 / (1 + exp(-n))
0.4

0.3

0.2

0.1

0
-5 -4 -3 -2 -1 0 1 2 3 4 5

You might also like