A SEMINAR ON

• ARTIFICIAL NEURAL NETWORK

\
 NEURAL NETWORK…
 What is a Neural Network?

 An Artificial Neural Network (ANN) is an information processing

paradigm that is inspired by the way biological nervous systems such
as the brain, processes information. The key element of this paradigm
is the novel structure of the information processing system. It is
composed of a large number of highly interconnected processing
elements (neurons) working in unison to solve specific problems.
ANNs, like people, learn by example. An ANN is configured for a
specific application, such as pattern recognition or data classification,
through a learning process. Learning in biological systems involves
adjustments to the synaptic connections that exist between the
neurons. This is true of ANNs as well.
 History of Neural Networks
 The study of the human brain is thousands of
years old. With the advent of modern
electronics, it was only natural to try to harness
this thinking process. The first step toward
artificial neural networks came in 1943 when
Warren McCulloch, a neurophysiologist, and a
young mathematician, Walter Pitts, wrote a
paper on how neurons might work. They
modeled a simple neural network with electrical
circuits.
 Why use neural networks?
• Neural networks, with their remarkable ability to derive meaning from
complicated or imprecise data, can be used to extract patterns and detect
trends that are too complex to be noticed by either humans or other
computer techniques. A trained neural network can be thought of as an
"expert" in the category of information it has been given to analyses. This
expert can then be used to provide projections given new situations of
interest and answer "what if" questions. Other advantages include:-

 Adaptive learning: An ability to learn how to do tasks based on the data given
for training or initial experience.

 Self-Organisation: An ANN can create its own Organisation or representation

of the information it receives during learning time.
 in monitoring:

– networks have been used to monitor

• the state of aircraft engines. By monitoring vibration levels and sound,
early warning of engine problems can be given.
 CONTD…
– British Rail have also been testing a similar application
monitoring diesel engines.
 Self-Organization: An ANN can create its own
organization or representation of the information
it receives during learning time.
 Real Time Operation: ANN computations may be
carried out in parallel, and special hardware
devices are being designed and manufactured
which take advantage of this capability.
 Fault Tolerance via Redundant Information
Coding: Partial destruction of a network leads to
the corresponding degradation of performance.
However, some network capabilities may be
retained even with major network damage.
ANALOGY BETWEEN NN & ANN:-
• Soma -> Processing unit

• Dendrites -> input layer

• Axon ->output layer

• Synapse -> weight

A NEURAL NETWORK(NN)
 COMPARISION BETWEEN COMPUTER
SYSTEMS & NEURAL NETWORKS
 computer systems are good at fast arithmetic.
 computers are also good at doing precisely
what the programmer programs them to do.
 but at the same time computer systems are not
so good at –
• interacting with noisy data.
• massive parallelism
• fault tolerism
• and adapting to circumstances
 CONTD..

BUT NEURAL NETWORKS ARE HELPFUL-

 where we can't formulate an algorithmic

solution.
 where we can get lots of examples of the
behavior we require.
 where we need to pick out the structure from
existing
 An engineering approach
A simple neuron
• An artificial neuron is a device with many inputs and one output. The neuron
has two modes of operation; the training mode and the using mode. In the
training mode, the neuron can be trained to fire (or not), for particular input
patterns. In the using mode, when a taught input pattern is detected at the
input, its associated output becomes the current output. If the input pattern
does not belong in the taught list of input patterns, the firing rule is used to
determine whether to fire or not.

A SIMPLE NEURON
 Firing rules
• The firing rule is an important concept in neural networks and
accounts for their high flexibility. A firing rule determines how
one calculates whether a neuron should fire for any input
pattern. It relates to all the input patterns, not only the ones on
which the node was trained.
• A simple firing rule can be implemented by using Hamming
distance technique. The rule goes as follows:
• Take a collection of training patterns for a node, some of which
cause it to fire (the 1-taught set of patterns) and others which
prevent it from doing so (the 0-taught set). Then the patterns
not in the collection cause the node to fire if, on comparison ,
they have more input elements in common with the 'nearest'
pattern in the 1-taught set than with the 'nearest' pattern in the
0-taught set. If there is a tie, then the pattern remains in the
undefined state.
 CONTD…
• For example, a 3-input neuron is taught to output 1 when the input
(X1,X2 and X3) is 111 or 101 and to output 0 when the input is 000 or
001. Then, before applying the firing rule, the truth table is;

X1: 0 0 0 0 1 1 1 1
X2: 0 0 1 1 0 0 1 1
X3: 0 1 0 1 0 1 0 1
OUT: 0 0 0/1 0/1 0/1 1 0/1 1

As an example of the way the firing rule is applied, take the pattern
010.
It differs from 000 in 1 element, from 001 in 2 elements, from 101 in 3
elements and from 111 in 2 elements. Therefore, the 'nearest' pattern is
000 which belongs in the 0-taught set. Thus the firing rule requires that
the neuron should not fire when the input is 001. On the other hand,
011 is equally distant from two taught patterns that have different
outputs and thus the output stays undefined (0/1).
 CONTD..
• By applying the firing in every column the
following truth table is obtained;
X1: 0 0 0 0 1 1 1 1
X2: 0 0 1 1 0 0 1 1
X3: 0 1 0 1 0 1 0 1
OUT: 0 0 0 0/1 0/1 1 1 1

The difference between the two truth tables is

called the generalization of the neuron.
Therefore the firing rule gives the neuron a
sense of similarity and enables it to respond
'sensibly' to patterns not seen during training.
 Architecture of neural networks
1. Feed-forward networks
Feed-forward ANNs allow
signals to travel one way only
from input to output. There is
no feedback (loops) i.e. the
output of any layer does not
affect that same layer. Feed-
forward ANNs tend to be
straight forward networks that
associate inputs with outputs.
They are extensively used in
pattern recognition. This type
Diagram of feed-forward of organization is also referred
to as bottom-up or top-down.
networks
 Contd..
2.Feedback networks
Feedback networks can have
signals traveling in both directions
by introducing loops in the
network. Feedback networks are
very powerful and can get extremely
complicated. Feedback networks are
dynamic; their 'state' is changing
continuously until they reach an
equilibrium point. They remain at the
equilibrium point until the input
changes and a new equilibrium needs
to be found. Feedback architectures
are also referred to as interactive or
Diagram of feedback recurrent, although the latter term is
often used to denote feedback
networks connections in single-layer
organizations.
 Neural Network layers:-

• The commonest type of artificial neural network

consists of three groups, or layers, of units: a layer of
"input" units is connected to a layer of "hidden" units,
which is connected to a layer of "output" units.
• The activity of the input units represents the raw
information that is fed into the network.
• The activity of each hidden unit is determined by the
activities of the input units and the weights on the
connections between the input and the hidden units.
• The behavior of the output units depends on the
activity of the hidden units and the weights between
the hidden and output units.
 CONTD..
• This simple type of network is interesting because the
hidden units are free to construct their own
representations of the input. The weights between the
input and hidden units determine when each hidden
unit is active, and so by modifying these weights, a
hidden unit can choose what it represents.
• We also distinguish single-layer and multi-layer
architectures. The single-layer organization, in which
all units are connected to one another, constitutes the
most general case and is of more potential
computational power than hierarchically structured
multi-layer organizations. In multi-layer networks, units
are often numbered by layer, instead of following a
global numbering.
 Pattern Recognition - an example
• An important application of neural networks is pattern
recognition. Pattern recognition can be implemented
by using a feed-forward neural network that has been
trained accordingly. During training, the network is
trained to associate outputs with input patterns. When
the network is used, it identifies the input pattern and
tries to output the associated output pattern. The
power of neural networks comes to life when a pattern
that has no output associated with it, is given as an
input. In this case, the network gives the output that
corresponds to a taught input pattern that is least
different from the given pattern.
 CONTD…
• Pattern recognition has long been studied in relation to
many different (and mainly unrelated) applications,
such as
• classifying galaxies by shape
• identifying fingerprints
• highlighting potential tumors on a mammogram
• handwriting recognition
• Human expertise in these and many similar problems
is being supplemented by computer-based
procedures, especially neural networks. Pattern
recognition is extremely widely used, often under the
names of `classification', `diagnosis' or `learning from
 AN EXAMPLE

• For example:
The network of figure 1 is trained to recognizes the
patterns T and H. The associated patterns are all
black and all white respectively as shown below.
• If we represent black squares with 0 and white
squares with 1 then the truth tables for the 3 neurons
after generalisation are;
TOP NEURON
X11: 0 0 0 0 1 1 1 1
X12: 0 0 1 1 0 0 1 1
X13: 0 1 0 1 0 1 0 1
OUT: 0 0 1 1 0 0 1 1
• MIDDLE NEURON
X21: 0 0 0 0 1 1 1 1
X22: 0 0 1 1 0 0 1 1
X23: 0 1 0 1 0 1 0 1
OUT: 1 0/1 1 0/1 0/1 0 0/1 0
• BOTTOM NEURON
X21: 0 0 0 0 1 1 1 1
X22: 0 0 1 1 0 0 1 1
X23: 0 1 0 1 0 1 0 1
OUT: 1 0 1 1 0 0 1 0
 Perceptrons

• The most influential work on neural nets in the 60's went under
the heading of 'Perceptrons' a term coined by Frank Rosenblatt.
The Perceptrons turns out to be an MCP model ( neuron with
weighted inputs ) with some additional, fixed, pre--processing.
Units labeled A1, A2, Aj , Ap are called association units and
their task is to extract specific, localized featured from the input
images. Perceptrons mimic the basic idea behind the
mammalian visual system. They were mainly used in pattern
recognition even though their capabilities extended a lot more.
 The learning process
• The brain basically learns from experience. The
memorisation of patterns and the subsequent
response of the network can be categorized into two
general paradigms:

 Associative mapping in which the network learns to produce

a particular pattern on the set of input units whenever
another particular pattern is applied on the set of input units

 Regularity detection in which units learn to respond to

particular properties of the input patterns. Whereas in
associative mapping the network stores the relationships
among patterns, in regularity detection the response of each
unit has a particular 'meaning'. This type of learning
mechanism is essential for feature discovery and knowledge
representation.
Every neural network posseses knowledge which is contained
in the values of the connections weights. Modifying the
knowledge stored in the network as a function of experience
implies a learning rule for changing the values of the weights
 Information is stored in the weight matrix W of a neural
network. Learning is the determination of the weights.
Following the way learning is performed, we can
distinguish two major categories of neural networks:

• Fixed networks in which the weights cannot be

changed, i.e. dW/dt=0. In such networks, the weights
are fixed a priori according to the problem to solve.

• Adaptive networks which are able to change their

weights,
i.e. dW/dt not = 0
 
 All learning methods used for adaptive neural
networks can be classified into two major categories:
 Unsupervised learning uses no external teacher and is
based upon only local information. It is also referred to as self-
Organisation, in the sense that it self-organises data presented
to the network and detects their emergent collective properties.
Paradigms of unsupervised learning are Hebbian learning and
competitive learning.

 Supervised learning which incorporates an external

teacher, so that each output unit is told what its desired
response to input signals ought to be. During the learning
process global information may be required. Paradigms of
supervised learning include error-correction learning,
reinforcement learning and stochastic learning.
An important issue concerning supervised learning is the
problem of error convergence, i.e. the minimisation of error
between the desired and computed unit values. The aim is to
determine a set of weights which minimizes the error. One well-
known method, which is common to many learning paradigms,
is the least mean square (LMS) convergence..
• Hebb's Rule: The first, and undoubtedly the best known,
learning rule was introduced by Donald Hebb.His basic rule is:
If a neuron receives an input from another neuron and if both
are highly active the weight between the neurons should be
strengthened.
• Hopfield Law: It is similar to Hebb's rule with the exception that
it specifies the magnitude of the strengthening or weakening. It
states, "If the desired output and the input are both active or
both inactive, increment the connection weight by the learning
rate, otherwise decrement the weight by the learning rate."
• The Delta Rule: This rule is a further variation of Hebb's Rule.
It is one of the most commonly used. This rule is based on the
simple idea of continuously modifying the strengths of the input
connections to reduce the difference (the delta) between the
desired output value and the actual output of a processing
element. This rule changes the synaptic weights in the way that
minimizes the mean squared error of the network. This rule is
also referred to as the Widrow-Hoff Learning Rule and the
Least Mean Square (LMS) Learning Rule.
 New Application areas:-
 Pen PC's
PC's where one can write on a tablet, and the writing will be
recognise and translated into (ASCII) text.

 Speech and Vision recognition systems

Not new, but Neural Networks are becoming increasingly
part of such systems. They are used as a system component,
in conjunction with traditional computers.

• As Neural Network chips become available, the possibility of

simple cheap systems which have learned to recognise simple
entities (e.g. walls looming, or simple commands like Go, or
Stop), may lead to their incorporation in toys and washing
machines etc. Already the Japanese are using a related
technology, fuzzy logic, in this way. There is considerable
interest in the combination of fuzzy and neural technologies.
 CONCLUSION
• The computing world has a lot to gain from neural networks. Their ability to
learn by example makes them very flexible and powerful. Furthermore there
is no need to devise an algorithm in order to perform a specific task; i.e.
there is no need to understand the internal mechanisms of that task. They
are also very well suited for real time systems because of their fast response
and computational times which are due to their parallel architecture.

• Neural networks also contribute to other areas of research such as neurology

and psychology. They are regularly used to model parts of living organisms
and to investigate the internal mechanisms of the brain.

• Perhaps the most exciting aspect of neural networks is the possibility that
some day 'conscious' networks might be produced. There is a number of
scientists arguing that consciousness is a 'mechanical' property and that
'conscious' neural networks are a realistic possibility.

• Finally, I would like to state that even though neural networks have a huge
potential we will only get the best of them when they are integrated with
computing, AI, fuzzy logic and related subjects.
THANK YOU…