Scribd
Upload
299 views7 pages

Backpropagation

This document discusses classification using backpropagation in neural networks. It explains that backpropagation is a neural network learning algorithm that was developed to build computational models of neurons. It works by adjusting the weights between units in a neural network so it can correctly predict the class labels of input data during training. While neural networks have strengths like tolerance to noise and ability to classify untrained patterns, they also have weaknesses like long training times and lack of interpretability.

Uploaded by

Shruti Jamsandekar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
299 views7 pages

Backpropagation

This document discusses classification using backpropagation in neural networks. It explains that backpropagation is a neural network learning algorithm that was developed to build computational models of neurons. It works by adjusting the weights between units in a neural network so it can correctly predict the class labels of input data during training. While neural networks have strengths like tolerance to noise and ability to classify untrained patterns, they also have weaknesses like long training times and lack of interpretability.

Uploaded by

Shruti Jamsandekar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 7

Classification by Backpropagation

 Backpropagation: A neural network learning algorithm


 Started by psychologists and neurobiologists to develop
and test computational analogues of neurons
 A neural network: A set of connected input/output units
where each connection has a weight associated with it
 During the learning phase, the network learns by
adjusting the weights so as to be able to predict the
correct class label of the input tuples
 Also referred to as connectionist learning due to the
connections between units
Neural Network as a Classifier
 Weakness
 Long training time
 Require a number of parameters typically best determined
empirically, e.g., the network topology or ``structure."
 Poor interpretability: Difficult to interpret the symbolic meaning
behind the learned weights and of ``hidden units" in the network
 Strength
 High tolerance to noisy data
 Ability to classify untrained patterns
 Well-suited for continuous-valued inputs and outputs
 Successful on a wide array of real-world data
 Algorithms are inherently parallel
 Techniques have recently been developed for the extraction of
rules from trained neural networks
A Neuron (= a perceptron)

- k
x0 w0

x1 w1
 f
output y
xn wn
For Example
n
Input weight weighted Activation y  sign( wi xi  k )
vector x vector w sum function i0

 The n-dimensional input vector x is mapped into variable y by


means of the scalar product and a nonlinear function mapping
A Multi-Layer Feed-Forward Neural
Network
Output vector

Errj  O j (1 O j ) Errk w jk


Output layer k

 j   j  (l)Errj
wij  wij  (l)ErrjOi
Hidden layer Errj  O j (1 O j )(Tj  O j )

wij 1
Oj 
1 eI j
Input layer
I j   wijOi  j
i
Input vector: X
How A Multi-Layer Neural Network Works?
 The inputs to the network correspond to the attributes measured
for each training tuple
 Inputs are fed simultaneously into the units making up the input
layer
 They are then weighted and fed simultaneously to a hidden layer
 The number of hidden layers is arbitrary, although usually only one
 The weighted outputs of the last hidden layer are input to units
making up the output layer, which emits the network's prediction
 The network is feed-forward in that none of the weights cycles
back to an input unit or to an output unit of a previous layer
 From a statistical point of view, networks perform nonlinear
regression: Given enough hidden units and enough training
samples, they can closely approximate any function
Defining a Network Topology
• First decide the network topology:
# of units in the input layer,
# of hidden layers (if > 1),
# of units in each hidden layer, and # of units in the output layer
• Normalizing the input values for each attribute measured
in the training tuples to [0.0—1.0]
• One input unit per domain value, each initialized to 0
• Output, if for classification and more than two classes,
one output unit per class is used
• Once a network has been trained and its accuracy is
unacceptable, repeat the training process with a different
network topology or a different set of initial weights
Backpropagation
 Iteratively process a set of training tuples & compare the network's
prediction with the actual known target value
 For each training tuple, the weights are modified to minimize the
mean squared error between the network's prediction and the
actual target value
 Modifications are made in the “backwards” direction: from the output
layer, through each hidden layer down to the first hidden layer, hence
“backpropagation”
 Steps
 Initialize weights (to small random #s) and biases in the network

 Propagate the inputs forward (by applying activation function)

 Backpropagate the error (by updating weights and biases)

 Terminating condition (when error is very small, etc.)

You might also like