Scribd
Upload
20 views12 pages

SC - Unit 2 - Hopfield Network

Uploaded by

teacher2
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
20 views12 pages

SC - Unit 2 - Hopfield Network

Uploaded by

teacher2
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 12

Soft Computing

Unit -2

 Hopfield Network
Hopfield Networks
 In 1982, John J. Hopfield introduced a neural network, which stores and retrieve information
from memory, in the same manner as the human brain. .
 Initially , the Hopfield network is trained to store a number of patterns and then it is used to
recognize any of the learned patterns, even if the information about the patterns is partial or
corrupted.
 The Hopfield networks work as content addressable memory systems and even when the
pattern served to the network is partial, it eventually settles down and returns the closest
guess of the pattern.
 Thus, this network model is stable in pattern recognition.
 This network has found many useful applications in associative memory and various
optimization problems.
Structure :
 A Hopfield network is a special type of recurrent neural network.
 It is a single layered network consisting of n fully connected neurons.
 There are no specific input or output nodes.
 All the nodes are inputs to each other, and these are output nodes as well.
 All nodes are connected to all others in both directions, withy equal weights in the two
directions.
 These are recurrent neural network, as output values are fed back to input in an undirected
way.
 A node is either on (active / firing) or off (inactive / not firing). Thus, nodes have binary
values for their states.
 A network with n nodes is capable of learning patterns of n-bits.
 Each node is assigned an activation state according to its threshold, using an activation
function.

 Activation function is defined as :

 n
1, if  Wij xS j   i
Ai   j 1

0, otherwise

W
Here, ij is the weight of the connection from X i and X j

S j is the state of X j (i.e. either 1 or 0- active or inactive) and

 i is the threshold of X i
Following properties hold true in a Hopfield Neural Network:

 The connections between two nodes are symmetric, i.e. Wij  W ji

 There is no connection with itself, i.e. Wii  0 fori

 Node is always in either active or inactive state.

 Positive weighted connections indicates that two nodes tend to activate each other.

 Negative weighted connections indicates that an active node can deactivate other connected

active nodes.
The Hopfield networks are categorised in following two categories :

 Discrete Networks
 These networks give any of the two discrete outputs. Based on the output
received, there are two types:
o Binary: In this type, the output is either 0 or 1.
o Bipolar: In this type, the output is either -1 (when output < 0) or 1 (When
output > 0)

 Continuous Networks
 Instead of receiving binary or bipolar output, the output value lies between 0
and 1.
 Energy Function in Hopfield Network
 In Hopfield networks, there are two different types of updations.
o Synchronous: Updating all the nodes simultaneously each time.
o Asynchronous: Updating only one node at a time. That node is selected randomly or
based on specific rules. In asynchronous updation, each state of Hopfield network is
associated with an energy value. The value is obtained from a function, and that function
is named an energy function. This function can decrease or remain unchanged during
updation. This energy function of the Hopfield network is defined as:
1 n n n n
Ef     yi y j wij   xi yi    i yi
2 i 1 j 1 i 1 i 1
j i
If the network is stable, then the above energy function decreases whenever the state of any
node changes.
Training Algorithm of Discrete Hopfield Network

 During training of discrete Hopfield network, weights will be updated.


 With discrete network, both binary input vectors as well as bipolar input vectors can be
used. Hence, in both the cases, weight updates can be done with the following relation:
Step 7 : Feed back the obtained output to all other units. Thus, the activation vectors are
updated.
Step 8: Finally, test the network for convergence.
 Continuous Hopfield Network :

 A discrete Hopfield net can be modified to a continuous model, in which time is


assumed to be a continuous variable.
 The nodes of the network have a continuous, graded output rather than a two-
state binary output.
 Thus, the energy of the network decreases continuously with time.
 The continuous Hopfield networks can be realized as an electronic circuit,
which uses non-linear amplifiers and resistors.
 Applications:
 The purpose of a Hopfield network is to store one or more patterns and to recall
the full patterns based on partial input.
 This network is used for complex problems of optimization and pattern
classification.
 Wide variety of constraint-satisfaction problems can be solved by Hopfield
network.
 Hopfield network can be viewed as a content addressable memory system. It is
trained for different patterns of the same size. Retrieval of full patterns is
possible even if we supply a portion of the pattern.

You might also like