Presentation on

Neural Networks.
Basics Of Neural Networks
• Neural networks refers to a connectionist model that
simulates the biophysical information processing occurring
in the nervous system.
• It can also be defined as an interconnected assembly of
simple processing elements ,units or nodes whose
functionality is loosely based on the animal neuron.
• And a cognitive information processing structure based
(on models of brain function. In a more formal engineering
context a highly parallel dynamical system with the
topology of a directed graph that can carry out information
processing by means of it's state response to continuous
or initial input.
Basics Of Neural Networks
Benefits Of Neural Networks
• Non-linearity
• Input-output Mapping
• Adaptivity
• Evidential Response
• Contentional Information
• Fault Tolerance

• RESEARCH THOUGHT
• 1. Neural networks are highly parallel structures which is true
because human brain functions in the same way.
• 2. But apart from being parallel it is has priority based
parallelism.
• 3. Apart from being parallel there is interaction between
these parallel processes and in the end one process may
dominate while others vanish or survive with much lower
priority.
Benefits Of Neural Networks
Facts
• 1. Knowledge is
acquired by the
network from its
environment
through a learning
process.
• 2. Interneuron
connection
strengths known as
synaptic weights
are used to store
the acquired
knowledge
LEARNING IN NEURAL
NETWORKS

LEARNING MAY BE DEFINED AS

• 1. To learn from environment and improve its performance
through learning.
• 2. Learning is a process by which free parameters of
neural networks are adapted through the process of
simulation by the environment in which the network is
embedded. The type of learning is as follows-
a. Neural network is simulated by environment.
b. Neural network undergoes change.
c. Neural network responds in a way to environment.
LEARNING IN NEURAL
NETWORKS
Types Of Learning
• a. Error correction learning
• b. Memory based learning
• c. Competitive learning
• d. Boltzman learning

RESEARCH THOUGHT
• 1. Normally learning process is iterative process in which
neural networks consistently learn from environment.
• 2. Neural networks must try for self eradication of
of error by heuristically moving towards the goal state.
• 3. It means that there should be combination of heuristic
knowledge and previous data to obtain the final result.
MEMORY BASED LEARNING
• Past experiences are explicitly stored in a large memory of correctly
classified input-output examples.
• xi is the input vector
• di denotes the desired response
• c1 and c2 are classification examples
• Retrieving and analyzing the training data by putting into classifications
c1 and c2.

RESEARCH THOUGHT
• 1. Since memory based learning is only a classification process it is
inaccurate because it does not account long term and short term
memory.
• 2. It should be a layered process where information if filtered from the
forward layers to the backward layers.
• 3. The forward layers are short term memory layers where as back
layers are long term memory layers.
• 4. The neural network must operate by considering all the layers giving
short term memory layers more priority than long term memory layers.
MEMORY BASED LEARNING
Memory Based Learning
(Working Example)
• In memory based learning there is classification of input-
output examples {(Xi,Di)}i=1 to N.where Xi is the input
vector and Di is the desired response.
• A working example of memory based learning is car
movement. We shall classify all the cases into two parts 1
(car speed up) and 0 (car slow down).The input signals
are X1 -> road conditions , X2-> traffic signal, X3-> fuel
efficiency , X4-> road ascent.
• Now when a set of inputs are applied to X1, X2, X3, X4
then the response is either speed up or slow down. As per
the memory based learning all these cases can be stored
during learning and new cases can then be classified as
being of either speeding up or slow down of the car
depending on the input conditions.
Memory Based Learning
(Working Example)
HEBBIAN LEARNING
• When an axon of cell a is near enough to cell b and
repeatedly takes part in firing it some growth process or
metabolic change takes place in one or both cells such
that efficiency of a as one of the cells firing b is increased.
• It means that two neurons on the either side of synapse
are activated simultaneously causing the strength of
synapse to increase.

RESEARCH THOUGHT
• Hebbian learning should be classifieds into two parts
• 1. A process in which there is gradual shift toward
strengthening of synapse if the input total synaptic weights
are below a threshold value.
• 2. If the synaptic weights inputs are above a threshold
value there is a fast shift in a single iteration.
HEBBIAN LEARNING
Hebbian Learning (Working
Example)
• Hebbian learning in mathematical terms can be expressed
by considering a synaptic weight Wkj of neuron k with pre-
synaptic and post-synaptic signals denoted by Xj and Yk.
If pre-synaptic and post-synaptic signals are synchronous
then there is increase in weight .The adjustment applied to
the synaptic weight Wkj at time step n is
• Δ Wkj(n) = F(Yk(n),Xj(n))
• A working example is introduction of traffic signal X1->Red
, X2-> Yellow and X3-> Green. We can observe that initial
slow down at red signal Y1(n) and initial startup at green
signal Y2(n) is slow but with time the response becomes
stronger and faster.
Hebbian Learning (Working
Example)
COMPETITIVE LEARNING

• In competitive learning the output

neurons of a neural network compete
among themselves to become active.
• In competitive learning a set of neurons
behave differently to a given set of
inputs.
COMPETITIVE LEARNING
BOLTZMANN LEARNING

• The neurons constitute a recurrent structure

and they operate in a binary manner since for
example they may be in +1 or-1 state.
• There is flipping of states depending on the
input.

RESEARCH THOUGHT
• Blotzmann learning puts the neurons in only
two states +1 and -1 whereas actually they
should take a number of states depending on
the set of inputs previous states.
BOLTZMANN LEARNING
Boltzman Learning (working
Example)
• Boltzman machine operates on the energy generated
when a signal moves from neuron j to neuron k. This
process continues till the system reaches thermal
equilibrium or the desired state.
• A working example can be a thermostat which keeps a
check on the heat energy being released in various
processes in a factory. The Boltzmann system can
gradually learn the amount of heat energy released during
all processes and then learn to adjust the weights
maintaining an optimal temperature. In fact it can
automatically guide the temperature maintenance all the
time.
P (change) = 1/1+exp (-∆E/Ti)
Boltzman Learning (working
Example)
• We can easily calculate it as-

• X1 (Temperature process P1) , X2

(Temperature process P2) , X3 (Temperature
process P3) , X4 (Temperature process P4)
may cause the final temperature reading Ti
which is compared with required temperature
Tj. Energy change ∆E can be calculated and
error correction applied automatically.
SUPERVISED LEARNING
• In conceptual terms supervised learning may be thought
of as neural network having knowledge of the environment
and using that knowledge to formulate the neural network
by input-output examples.

RESEARCH THOUGHT
• 1. Supervised learning should be object based in which
we try to learn about an object from the environment.
• 2. It means there is need to first learn about the object
properties and then about the object methods.
• 3. Once the object has been learned neural network may
simulate it for a set of inputs
SUPERVISED LEARNING
Supervised Learning (Working
Example)
• The aircraft control system can become a good
example of supervised learning because the
aircraft navigation system faces new
environmental conditions all the time. But these
conditions are fed to the GPS, Ground support
and other devices which teach the system to
deal with them.
• The system can do the error correction learning
to stay on course and learn to manage the
system. When the system has fully learned to
automate itself, it can be put onto a pilot less
vehicle for self navigation with minimal outside
help.
Supervised Learning (Working
Example)
UNSUPERVISED LEARNING
• In unsupervised learning there is no external teacher.
Rather the process is made for a task-independent
measure of quality of representation that the network is
required to learn.
• Various stochastic methods like standard deviation
regression are used to obtain useful information from data.

RESEARCH THOUGHT
• Since there is no supervision required and all data is
collected and then analyzed it would be useful to first
create a broad classification of environment
• Once the environment has been classified data from the
environment can be further classified to make the data
collected to be more meaningful.
UNSUPERVISED LEARNING
Unsupervised Learning
(Working Example)
• In case of unsupervised learning there is no
error correction support applied. The data is
statistically classified into one or more classes.
• Unsupervised learning can be used in weather
forecast system. The data can be collected in
the form of variable values T (Temperature
Conditions), C (Cloud formations), H (Humidity
reading in and around a place), A (Air flow
readings).
• These can then use unsupervised learning to
learn to make a correct weather forecast as an
output of the neural network.
Unsupervised Learning
(Working Example)
SINGLE LAYER PERCEPTRON

• A perceptron is the simplest form of a neural network used

for the classification of patterns which are linearly
separable. It consists of a single neuron with adjustable
synaptic weights.
• Perceptron convergence theorem tries to do pattern
classification with only two classes in case of single layer
perceptron.

RESEARCH THOUGHT
• Single layer perceptron should be clocked as being a
slower and a faster neuron.
• Further the weights themselves should be a function of
time and should depend on delta t.
SINGLE LAYER PERCEPTRON
Single Layer Perceptron
(Working Example)
• Single layer Perceptron does binary classification and
then does error correction as per the learning rule by
modification of weights.
• An example can be a Perceptron that calculates the price
of a product. We can consider the input variable with
some initial weights X1(Market demand), X2(Input
material prices), X3(Past growth) X4(Profit expected). This
can be expressed as a linear equation.

• 1.2354 X1 + 2.3338 X2 + 6.4523 X3 + 1.1 X4 = Price

• Now single layer perceptron can be made to calculate the

exact price after error correction done by comparing the
output price with the actual price. Eventually it can predict
the correct price.
Single Layer Perceptron
(Working Example)
MULTI LAYER PERCEPTRON
• A multi layer perceptron consists of a sensory units that
constitute an input layer, one or more hidden layers of
computation nodes and an output layer of computation
nodes.
• Learning takes place using error-correction learning rule.
• The function used is a non linear activation function called
the activation function.

RESEARCH THOUGHT
• As there are numerous layers in multi layer perceptron
they should be classified as-
• 1. Which layer is faster than others?
• 2. Which layer has a higher priority?
• 3. Which layer is responsible for what part of output?
Multi-Layer Perceptron
(Working Example) XOR
RADIAL BASIS FUNCTION

• RBF looks at multi layer perceptron from curve

fitting point of view.
• RBF does complex pattern classification
• Task- A complex pattern classification problem
is cast in a high dimension space non-linearity
is more likely to be linearly separable than in
low dimensional space.
• Interpolation is the technique used for curve
fitting from data movement across the layers.
RADIAL BASIS FUNCTION
Radial Basis Function (Working
Example)
• In multi-layer perceptron calculation is done for an
approximate function to various layers of the multi-layer
perceptron. A function Φ(x) which approximates
movement of signal from one layer to another.

• An example can be application of RBF to study growth of

disease in a given population infecting people in a phased
manner. For example a disease starts by infecting 10% of
the population in 5 cities in the beginning. In the next
phase it grows by 5% in 10 more cities and grows to 20%
in the first 5 cities. This process can be approximated
using RBF and another RBF can calculate the move
against the spread of the disease.
Radial Basis Function (Working
Example)
SUPPORT VECTOR MACHINES
• Support vector machine is a linear machine which consists of a
decision surface in such a way that the margin of separation between
positive and negative cases is maximized.
• It follows the method of structural risk minimization an induction
principle based on the fact that the error rate of a learning machine on
test data is bounded by the sum of training error rate and a term that
depends on vapnik-chervonenkis dimension.

SVM supports the following three types of learning machines

• 1. Polynomial learning machine.
• 2. RBF function networks.
• 3.2-layer perceptrons.

RESEARCH THOUGHT
• SVM can calculate the probability of each point being a part of
classification.
• It can further deduce the results as to validity of each input for a
classification.
SUPPORT VECTOR MACHINES
Support Vector Machines
(Working Example)
• Support Vector machines use the hyper plane equation to
separate the examples into two classes +1 or -1. Support
vector machines use training data { (Xi,di) }i=1 to N. di =
+1 or di = -1 is the desired response from the neural
network. It uses equations

Wt Xi + b >=0 for di=+1

Wt Xi + b < 0 for di= -1

• An example can be a neural net which computes whether

a person can be given a loan (+1) or may not be given a
loan (-1). The input vector consists of the inputs Xi
(Income, past transactions, job type, family) etc. Support
vector machine can calculate optimal values of these
support vectors and then give the desired response
Support Vector Machines
(Working Example)