Proceedings of

the 2nd Intemational Workshop on

Autonomous DecentralizedSystem

Analysis and Study of Perceptron to Solve XOR Problem

Zhao Yanling, 1)eng Bimin, Wang Zhanrong

Southwest Jiaotong University. Chengdu 610031,Sichuan, China
E-mail: [email protected]

Abstract emphasizes on associative memory and optimization, etc.

This paper tries to explain the network structures and And the former can be further classified as multi-layer
methodk of single-layer perceptron and multi-layer perceptron, single-layer perceptron and functional link.
perceptron. It also analyses the linear division and Among them perceptron plays an important role in the
un-division problems in logical operation performed by problem of linearity divisibility and non-linearity
single-layer perceptron. XOR is linear un-division divisibility.
operation, which cannot be treated by single-layer As we all know, according to Boolean logic, especially
perceptron. R t h the analysis, several solutions are in duality calculation, there are operators as “AND, “ O R ,
proposed in the paper to solve the problems qf XOR. “NOT”, ‘‘NOT.A”’, ‘TJO.TOR, “XOR’ and
Single-layer perceptron can be improved by multi-layer “X.NOT.OR, and only “ X O R and “X.NOT.OR” are
perceptran. firnctional perceptron or quadratic ftmction. linearity non-divisible. It has been a bot issue to
These solutions are designed and analyzed. materialize and constmct the two operators in Neural
Network. The limitations of perceptron are discussed in
reference [2]. The functional combining net strucme is
1. Introduction introduced in references [3] [4]. In this paper, the authors
fmt give an introduction of networking and methods of
Artificial Neural Network (ANN)[l], constructed by the multi-layer perceptron and single-layer perceptron, and
human beings based on a knowledge of Neural Networks analyse the problem of linearity divisibility in the way of
oftheir brains,’is a functional Neural Network, which is an logical operation by perceptron, and then propose, with
information handling system takiig after the structure and some inter-comparisons, several solutions on how to
functions of buman brain. Compared with John Von utilize the perception to solve XOR problem.
Neumann’s series computer, it has higher parallelism,
stronger non-linear mapping ability, better error tolerance 2. Perceptron
and association memory function and greater self-leaming
ability. ANN bas been widely used in mode recognition, 2.1 Single-layer perceptron
information handling, associative memoly, :solution
optimizing and self-adapting control. As for the network structure of single-layer perceptron,
ANN can be structurally divided into two categories: see the following figure 1.
feedforward Neural Network and feedback Neural Among these: X ~ ( ; = I , ~ , .represents
. . . ~ ) the input of the
Network. The former is mainly adopted in the field of neuron i at the input level; y represents the output of
system discriminating and mode recognition, etc; the latter the neuron at the output level; 0, represents the

168
0 -7803 -7624 -2/02/$17.00 02002 IEEE
connection progression value of the neuron i at the input ( 5 ) Shift to (2) until progression value and the threshold
value keep the same to all the learning samples, that is the
convergence value is got.
Y t 2.2 Multi-layer perceptron

See the following network diagram of multi-layer

perceptron:

Fig. 1 Single-layer perceptron

level and the neuron at the output level.

The output of the neuron fills the formula (I), (2).

Y= rcz rU.5 - 0)
i 4
(1)

Among these, f ( ) is the activation function. Here

we pick unit function or the sign function S g n ( ) or S
-
function Sigmoid ( ) ; 0 is the threshold value.
The learning algorithm of the single-layer perceptron: Fig. 2 Multi-layer perceptron
(1) Put the preliminary value: make t = 0 , give
w,(t)(i = 1,2;.., n) and @(t) a smaller ram value
Amongthese, xi ( i = 1,2;..,n ) indicatesinput ofthe
(;to) respectively.
No. i neuron at the input level. z j ( j = 1,2;..,n),
(2) hput a learning sample X = ( . Y ~ , x ~ ; ~ ~T, x ~ )
output of No. j neuron at the concealed level. y ,
and its expected output d ,
neuron output at the output level. oij, connection
(3) Account the actual output and error:
progression value between the No. j neuron at the
concealed level and the No. i neuron at the output
neuron. vj represents the connection progression value
of the neuron j at the concealed level and the neuron at
e = (d - y)* the output level.
(4) Amend all the progression value and the threshold Neuron output at various levels satisfies following
value. formulas: (3). (4), ( 5 )
(3)
a,(t+l)= ~ ~ ( t ) + q ( d - y ) y ( l - y ) x ,

( i =1,2;..,n) y=
fl: 3
Cv,r,+B
(4)

0(t f 1) = 4 t )+ q(d - Y ) Y ( ~- Y )
Among these 7 is the learning step width which picks
the number between 0-1. 3.Perceptron classification and XOR problem

169
 In order to be general, we’ll think about the pei-ceptron XOR is a typical problem in linear un-division. What
of single neuron in the section 2.1, XOR means in logical operation is: when the two inputs
The adjustment of the connection progression value can are the same 1 or the same 0 in binary system, the output is
make perceptron’s reaction to a group of vectors achieve 0; when the two inputs are 1 or 0 respectively, the output is
the objective output of 0 or 1. This can be explained by 1. XOR requires dividing the four points in a plane with a
making diagrams in input vector space. Picking two input straight line as indicated in figure 4. Obviously it is
variables, x, , x, ,we can get figure 3: impossible. In general condition, it is impossible to solve
The condition wlxl + w2x2- 8 t 0 will divide the the problem of XOR with regular single-layer perceptron.
input plane into two parts. When the connection
progression .value and the threshold value change, the 4. Analysis of perceptron solution to solve
dividing line will move or turn round, but still keeps in a XOR problem
straight line. The threshold value divides the space of
vectors into several areas, which enable the perceptron to According to analysis above, the performance and
learning ability of single-layer perceptron is limited. In
be capable of classifying the input vectors and input
general condition, the simple single-layer perceptron can’t
samples. Perceptron cannot realize the corresponding
realize XOR,but only the multi-layer perceptron can solve
relation between the input and the output arbitrarily, or it
the problem of XOR. In order to solve the problem of
can’t solve the arbitray logical operation. It can only solve
XOR, we propose several solutions: multi-layer
the problem of the linear division mentioned in ligure 4,
perceptron, functional link perceptron, and single-layer
that is to say, preceptron cannot classify the problem of
perceptron can be improved by quadratic function.
linear un-division

\ tx2
4.1 Multi-layer perceptron to solve XOR problem

It is impossible to solve the problem of XOR within the

limitation of ordinary single-layer perceptron. The
simplest and most effective way is to add hiding layers to
change the single-layer net shucture into multi-layer net
shucture by using multi-layer perceptron. Here we will list
’ w+, + 0 2 x 2 -e <o some solutions. We take the output of each layer as in the
formula in section 2.1, namely:
Fig. 3 Input vector plane map

y = I ( C wizi- 8 )
i= I

yy O(1,l)
2
zi = f ( Z v j x j - O j )
id

.-+\- 0
0
Fig. 4 XOR problem sketch map n is the number of hiding layer of neuron.

170
 4
Solution. I

XI Xl
Fig. 6XOR percepuon of new o,o,B,

Solution 3

Y
-
0
1
0. 1
0

-v
Fig. 5 XOR'operation perceptron

Solution 2

0 1 .1

1 1 1 1 0

Fig. 7 Perceptron of one hiding neuron

171
 Solution 4 functional combining perceptron, many problems can be
solved by single-layer net. Figure 9 shows:
XI x2 Y
A
0 0 0 0 0
0 1 0 I O
1 0 1 0 0
1 1 1 1 1

XI x2 XI ‘ X 2

Fig. 9 Functional combnating network

On the basis of simple single-layer perceptron, the

introduction of xI . x 2 means that one input is added,
and this input is not new variable in nature but the
production of the mutual influence or encouragement of
xI and x 2 .So the output of this net is

t
XI
Fig. 8 Perceptron of three hiding neurons
T
X2
= f ( x , ~+lx 2 0 1 + X,~,OJ,
It can solve the problem of XOR. In fact, make
o1= w 2 =1,w3 = - 2 , e = I , a n d w e h a v e :
y = f ( x , + x* - 2x,x, - 1)
-e)

Rotating 45” ,the coordinate is changed to be a new one:

In solution 4, there are three neurons in the middle layer,
xI,x2 , the quadratic curve xI + X , - 2x,x2 = 1
. I

is
in theory, there is no limit on the number of iieurons
( n 2 1). However, it is unnecessary as the number will changed to be:
increase working load and has a direct impact on the speed
of constringency. which results in making the task more
difficult to fulfill. In fact, solution 1,2,3 is the simplified
It’s a hyperbola. It inputs two groups of models into ((O,O),
forms of solution 4.
(1.1)) ((l,O), (0,1)] and divides into two types: “0” type
In addition, the activation h c t i o n f , the threshold
and “ I ” type.
value 0 ~ e, and the weight wij are adjustable in the
previous four solutions. That is to say, there are a lot of
4.3 Quadratic function perceptron
combination projects that will be enumerated there.

The structures of quadratic function perception are

4.2 Functional combining perceptron
similar to the single-layer perceptron, while the differences
lie in which the activation fimction is quadratic function.
In some references [3] [4], the item of high rank is
Output of neural network should be satisfied with the
introduced, called Functional Combination. Through

172
following formula: XOR is linear un-division operation that cannot be
treated by the novel single-layer perceptran. Quadratic
y= f
[ COiXi
i:,
-0
] function perceptron is capable of learning XOR problems.
The essence of this improved quadratic function
perceptron lies in the following fact: neuron activation
f ( A= P 2
function employing quadratic function to replace unit
In quadratic function perceptron, the output expected value
function (or Sigmoid function), select expectation and the
is zero and big enough value L. To the problem of duality
most optimized learning step width. Simulation is also
XOR,we value L=l. Let’s assume the learning algorithm feasible by utilizing other functions forms.
as the follows:
(1) Make it originate: Let f =0 , give 5.Conclusion
w j ( t ) ( i= 1,2;..,n) and O ( f ) each a smaller random
value which isn’t zero. XOR is linear un-division operation, which cannot be
l- treated by single-layer perceptron. Six solutions are
(2) In put a learning sample X = (x,,x2;..,xn)
proposed to solve the problem of XOR.Four in which are
and its expected output “ d ”.
realized by means of multi-layer perceptron. General
(3) Calculate the actual output expressions have been provided, threshold value and
activation function are also adjustable, many potential
solutions are possibly involved. Multi-layer neural
network can always solve the problems of XOR or
(4) Modify each right value and threshold value “X.NOT.OR, and can implement any elements Boolean
When d =0: function and logical calculations, but at the cost of
w,(t+ 1) = wi(t)- q&xi (i = 1,2;..,n) complication of the system. The functional perceptron and

8(r + 1) = B ( f ) - 7&
quadratic function perceptron used to solve the problems
of XOR belong to improved sing-layer perceptron which
When d = L :
are characterized by more powerful learning functions and
wi ( t + 1) = wj( t ) + r]&xj (i = 1,2,...,n) faster convergence speed, compared with traditional
s(t + 1) = ~ ( t+ )q& single-layer perceptron.

Here r] is learning step width.

Turn ( 5 ) to (2), till the neural network is stable and References
unchanged for all learning samples.
[ I ] Jinfan, Neuml ComputationalIntelligenceBaric, Southwest
From the stIuctures and learning algorithm, we know:
Jiaotong University Press, 2000.
when d = 0 , each learning process of the quadratic
[21 Congshuang, “The Analysis, Limitations and Enlargement of
function perceptron is a process of getting crest value
perceptron”,Automation Panorama, No.3, 2000.pp. 34-36.
point of the quadratic function. The best step width we
[31 Harunei. “Discussion of XOR Roble”’. Journal of Hebei
choose is:
University, Vo1.19,No. I, Mar. 1999, pp. 24-27.
1 [41 Li Hongxing, “MathematicalNeural Nehvorks(III)”, Journal
r]= ”
of Beiiiig Normal University(NaNral Science), Vo1.33. N0.3.Sep.
l+CX,?
i=l 1997,pp. 305-311.

173