Scribd
Upload
44 views8 pages

Task On Hopfield Networks

The document discusses Hopfield Networks, a type of recurrent neural network used for associative memory, detailing their structure, weight updating mechanisms, and learning processes. It provides a specific example of a 3-unit Hopfield network trained on three vectors, including a step-by-step procedure for establishing a weight matrix and evaluating a test case. The document concludes with a solution to an assignment question related to the network's operation.

Uploaded by

pysingh289
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
44 views8 pages

Task On Hopfield Networks

The document discusses Hopfield Networks, a type of recurrent neural network used for associative memory, detailing their structure, weight updating mechanisms, and learning processes. It provides a specific example of a 3-unit Hopfield network trained on three vectors, including a step-by-step procedure for establishing a weight matrix and evaluating a test case. The document concludes with a solution to an assignment question related to the network's operation.

Uploaded by

pysingh289
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

NPTEL

Video Course on Machine Learning

Professor Carl Gustaf Jansson, KTH

Topic: Hopfield Networks

Solution and Explanations for Assignment Q 11 - Week 6


Assignment tasks - Week 6 2021

Problem # 11 Correct Marks: 2 Theme: Hopfield Networks and Boltzman Machines

A 3 unit Hopfield network is trained with the following 3 vectors:


V1 = 1 1 1 V2 = 1 -1 1 V3= -1 -1 -11 , N=3 Stable unit states = -1, 1 Test case 1 0 0

Procedure for establishing a weight matrix and for calculation of a new of a test case using a matrix approach:
1. Define X with the three training vectors as rows
2. Define the weight matrix W = XTX
3. Normalise weight matrix W by dividing with N (number of training vectors)
4. Evaluate test case by multiplying test case with the normalized W.
5. Apply a threshold function with threshold = 0 to all elements in resulting vector.

1 1 -1
- 1 1 1 3 1 3 1 1/3 1
XT = 1 -1 -1 1. X = 1 -1 1 2. 1. XTX = 1 3 1 3. 1/3 1 1/3
1 1 -1 -1 -1 -1 3 1 3 1 1/3 1

S = 1 0 0 * 1 1/3 1 What is the resulting state vector ?


1/3 1 1/3
Answer D 1 1/3 1 A. 1 -1 1 B. 1 1 -1 C. -1 1 1 D. 1 1 1
Hopfield Networks
A Hopfield network :
- is one approach to the realization of associative memory
- is an instance of an auto-association memory
- is considered a recurrent neural network (RNN), even if it is not able to handle temporal sequences,
but it has states and cycles in the network
- is a one layer neural network in the sense that all units are input/output units
- is a fully-connected neural network with symmetric weights.
- has units modelled as inspired by the Mcculloch and Pitts neuron model.
- can have its states updated in a syncronous as well as an asyncronous mode
- has weight updating mechanisms based upon Hebbian learning
- has an Energy concept that ensures a convergence towards a stationary state.
- the model enables fix point stable attractors.

Hopfield networks were invented by John Hopfield in 1982.


Updating Units in a Hopfield Network
´One shot learning´ : Assuming that X is a matrix with N dataitem vectors as rows
The weight matrix W = 1/N * (XT X)

Updating the state of one unit in the Hopfield network is performed using the following rule:

Si = +1 if Sum j Wij * Sj >= threshold for unit i otherwise -1


where:
• Wij is the strength of the connection weight from unit j to unit i.
• Sj is the state of unit j

Thus, the values of neurons i and j will converge if the weight between them is positive. Similarly, they will
diverge if the weight is negative.

Updates in the Hopfield network can be performed in two different ways:


• Asynchronous: Only one unit is updated at a time. This unit can be picked at random, or a pre-defined order
can be imposed
• Synchronous: All units are updated at the same time. This requires a central clock to the system in order to
maintain synchronization. This method is viewed by some as less realistic, based on an absence of observed
global clock influencing analogous biological or physical systems of interest.
Hopfield Network Example (1)
1. Patterns to Remember 3. Build Network
p1 p2 p3
1/3
1 2 1 2 1 2 1 2
1/3 [-]
3 4 3 4 3 4 -1/3
-1/3 [+]
1/3
3 4
2. Hebbian Weight Init: -1
Avg. Correlations across 3 patterns
p1 p2 p3 Avg 4. Enter Test Pattern

W12 1 1 -1 1/3 1/3


W13 1 -1 -1 -1/3 1 2 1/3
W14 -1 1 1 1/3 -1/3 -1/3
3 4
1/3
W23 1 -1 1 1/3
W24 -1 1 -1 -1/3 -1
+1 0 -1
W34 -1 -1 -1 -1
Threshold = 0
Hopfield Network Example (2)
Synchronous Iteration (update all nodes at once)

W = 1 1/3 -1/3 1/3 Test case S = 1 0 0 -1


1/3 1 1/3 -1/3 Si = +1 if Sum j Wij * Sj >= 0 otherwise -1
-1/3 1/3 1 -1
1/3 -1/3 -1 1
Si = 1 1 1 - 1

1/3
p1
1/3
-1/3 -1/3 = 1 2
1/3 3 4

-1
Stable State
Hopfield Network Example (3) Matrix calculation to retrieve state
V1 = 1 1 1-1 V2 = 1 1 -1 1 V3 = -1 1 1 -1 N=3 Stable unit states = -1, 1 Testcase = 1 0 0 1

Procedure for establishing a weight matrix and for calculation of a new of a test case using a matrix approach:
1. Define X with the three training vectors as rows
2. Define the weight matrix W = XTX
3. Normalise weight matrix W by dividing with N (number of training vectors)
4. Evaluate test case by multiplying test case with the normalized W.
5. Apply a threshold function with threshold = 0 to all elements in resultivng vector.

1 1 -1 3 1 -1 1 1 1/3 -1/3 1/3


1 1 1 1 1 1 -1 1 3 1 -1 1/3 1 1/3 -1/3
XT = 1 -1 1 1. X = 1 1 -1 1 2. XTX = -1 1 3 -3 3. -1/3 1/3 1 -1
-1 1 -1 -1 1 1 -1 1 -1 -3 3 1/3 -1/3 -1 1

Test 1 0 0 -1 1 1/3 -1/3 1/3 - > 2/3 2/3 2/3 -2/3 -> S = 1 1 1 -1
1/3 1 1/3 -1/3 Retrieved state
-1/3 1/3 1 -1
1/3 -1/3 -1 1
Solution to Q11

A 3 unit Hopfield network is trained with the following 3 vectors

V1 = 1 1 1 V2 = 1 -1 1 V3= -1 -1 -11 , N=3 Stable unit states = -1, 1 Test case 1 0 0

1 1 -1
- 1 1 1 3 1 3 1 1/3 1
XT = 1 -1 -1 1. X = 1 -1 1 2. 1. XTX = 1 3 1 3. 1/3 1 1/3
1 1 -1 -1 -1 -1 3 1 3 1 1/3 1

S = 1 0 0 * 1 1/3 1 = 1 1 1
1/3 1 1/3
1 1/3 1

You might also like