Scribd
Upload
72 views16 pages

Neural Networks (Review) : Piero P. Bonissone GE Corporate Research & Development

This document provides an overview of neural networks and their various types and architectures. It discusses supervised and unsupervised neural networks including multilayer perceptrons, radial basis function networks, and self-organizing maps. Perceptrons, ADALINE networks, backpropagation, and improvements to learning such as momentum are described. Examples include single layer networks for classification tasks and how multilayer perceptrons can represent increasingly complex decision boundaries.

Uploaded by

vani_V_prakash
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
72 views16 pages

Neural Networks (Review) : Piero P. Bonissone GE Corporate Research & Development

This document provides an overview of neural networks and their various types and architectures. It discusses supervised and unsupervised neural networks including multilayer perceptrons, radial basis function networks, and self-organizing maps. Perceptrons, ADALINE networks, backpropagation, and improvements to learning such as momentum are described. Examples include single layer networks for classification tasks and how multilayer perceptrons can represent increasingly complex decision boundaries.

Uploaded by

vani_V_prakash
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 16

Soft Computing: Neural Networks

1
Neural Networks
(Review)
Piero P. Bonissone
GE Corporate Research & Development
[email protected]
(Adapted from Roger Jang)
Soft Computing: Neural Networks
2
Outline
Introduction
Classifications
Amount of supervision
Architectures
Output Types
Node Types
Connection Weights
Reflections
Soft Computing: Neural Networks
3
Neural Nets: Classification
Supervised Learning
Multilayer perceptrons
Radial basis function networks
Modular neural networks
LVQ (learning vector quantization)
Unsupervised Learning
Competitive learning networks
Kohonen self-organizing networks
ART (adaptive resonant theory)
Others
Hopfield networks
Soft Computing: Neural Networks
4
Supervised Neural Networks
Requirement:
known input-output relations
input pattern output
Soft Computing: Neural Networks
5
Perceptrons
-Rosenblatt: 1950s
-Input patterns represented is binary
-single layer network can be trained easily
-output o is computed by
where
w
i
is a (modifiable) weight
x
i
is the input signal
is some threshold
f(
.
) is the activation function
o f w x
i i
i
n

_
,


1
f x x ( ) sgn( )

'

1
0
if x > 0
otherwise
Soft Computing: Neural Networks
6
Single-Layer Perceptrons
Network architecture
x1
x2
x3
w1
w2
w3
w0
y = signum(wi xi + w0)
Soft Computing: Neural Networks
7
Single-Layer Perceptron
Example: Gender classification
h
v
w1
w2
w0
Network Arch.
y = signum(hw1+vw2+w0)
-1 if female
1 if male
=
y
Training data
h (hair length)
v

(
v
o
i
c
e

f
r
e
q
.
)
Soft Computing: Neural Networks
8
Perceptron
Learning:
select an input vector
if the response is incorrect, modify all weights
where
t
i
is a target output
is the learning rate
If there exists a set of weights, then the method is
guaranteed to converge
w t x
i i i

Soft Computing: Neural Networks
9
XOR
Minsky and Papert reported a severe shortcoming
of single layer perceptrons, the XOR problem
not linearly separable
which (together with a lack or proper training
techniques for multi-layer perceptrons) all but killed
interest in neural nets in the 70s and early 80s.
x1 x2 output
0 0 0
0 1 1
1 0 1
1 1 0
x
x O
O 1
1 0
0
Soft Computing: Neural Networks
10
ADALINE
Single layer network (proposed by Widrow and Hoff)
Output is a weighted linear combination of weights
The error is described as
(for pattern p)
where
t
p
is the target output
o
p
is the actual output
o w x w
i i
i
n

0
1
( )
E t o
p p p

2
Soft Computing: Neural Networks
11
ADALINE
To decrease the error, the derivative with
respect to the weights is taken
The delta rule is:
( )

p i p p i
w t o x
( )

E
w
t o x
p
i
p p i
2
Soft Computing: Neural Networks
12
Multi-Layer Perceptrons
-Recall the output
-and the squared error measure
which is approximated by
-and the activation function
then the learning rule for each node can be derived
using the chain rule...
o f w x
i i
i
n

_
,


1
( )
E t o
p p p

2
( )
p
k
p
k
p
k
k
n
E t o

2
1
( )
f x
e
x

+

1
1
Soft Computing: Neural Networks
13
Multilayer Perceptrons (MLPs)
Learning rule:
Steepest descent (Backprop)
Conjugate gradient method
All optim. methods using first derivative
Derivative-free optim.
Network architecture
x1
x2
y1
y2
hyperbolic tangent
or logistic function
Soft Computing: Neural Networks
14
Backpropagation
to propagate the error back through a multi-
layer perceptron.
w
E
w
ki
p
ki
p

Soft Computing: Neural Networks


15
Improvements
Momentum term
smoothes weight updating
can speed learning up
w E w
w prev
+
Soft Computing: Neural Networks
16
MLP Decision Boundaries
A B
B A
A
B
XOR Interwined General
1-layer: Half planes
A B
B A
A
B
2-layer: Convex
A B
B A
A
B
3-layer: Arbitrary

You might also like