Scribd
Upload
5 views30 pages

Feed Back

Uploaded by

patelrahul099621
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
5 views30 pages

Feed Back

Uploaded by

patelrahul099621
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 30

ANALYSIS OF Applications of Artificial

FEEDBACK NEURAL Intelligence and Neural


Networks(CS-538)
NETWORKS
FEEDBACK NEURAL NETWORKS
These networks have connections that loop back, allowing information to be fed back into
the network.

These kinds of networks can have signals travelling from both directions, that is, from
input to output as well as from output to input.
Feedback neural networks, or RNNs, are characterized by their ability to maintain a state
that captures information about previous inputs. This is achieved through recurrent
connections that loop back from the output to the input of the same layer or previous
layers.
ASSOCIATE MEMORY
NETWORK
❑These kinds of neural networks work on the basis of pattern association, which
means they can store different patterns and at the time of giving an output they can
produce one of the stored patterns by matching them with the given input pattern.
❑These types of memories are also called Content-Addressable Memory CAM.
❑Associative memory makes a parallel search with the stored patterns as data files.
❑Following are the two types of associative memories we can observe −

1. Auto Associative Memory


2. Hetero Associative memory
AUTO ASSOCIATIVE MEMORY
NETWORK
Auto-associative memory networks are types of neural networks designed to store
and retrieve patterns.
It is a system that “associates” two patterns (X, Y) such that when one is
encountered, the other can be recalled.
An auto-associative memory network, also known as a recurrent neural network, is a
type of associative memory that is used to recall a pattern from partial or degraded
inputs.
EXAMPLE
HETERO ASSOCIATIVE MEMORY
A hetero-associative memory network is a type of neural network that stores
relationships between two different sets of patterns.
Unlike auto-associative networks (which associate patterns with themselves),
hetero-associative networks learn to map one set of patterns (inputs) to a different set
of patterns (outputs) i.e the input training vector and the output target vectors are not
the same.
TRAINING ALGORITHM
TESTING ALGORITHM
HOPFIELD NETWORK
❑A Hopfield Network is a type of recurrent artificial neural network invented by John
Hopfield in 1982.
❑It is primarily used for associative memory and optimization tasks.
❑The network consists of a single layer of neurons, where each neuron is connected to
every other neuron except itself. These connections are bidirectional and symmetric.
❑A Hopfield network which operates in a discrete line fashion or in other words, it can
be said the input and output patterns are discrete vector, which can be either binary
0,1or bipolar +1,−1in nature.
❑The network has symmetrical weights with no self-connections wij = wji and wii =
0.
❑The output of each neuron should be the input of other neurons but not the input of
self.
HOPFIELD NETWORK
TRAINING RULE
▪During training of discrete Hopfield network, weights will be updated. As we know that
we can have the binary input vectors as well as bipolar input vectors. Hence, in both the
cases, weight updates can be done with the following relation.
▪For storing a set of input patterns S(p) [p = 1 to P], where S(p) = S1(p) … Si(p) … Sn(p),
the weight matrix is given by:
Case 1 − Binary input patterns(0,1) :
Weight Matrix is given by

Case 2 − Bipolar input patterns(1, -1):


EXAMPLE
Discrete Hopfield Network with the bipolar representation of the input vector as [1 1
1 -1] /[1 1 1 0] (in case of binary representation). Test the Hopfield network with
missing entries in the first and second components of the stored vector (i.e. [0 0 1
0]).
Solution: input vector, x = [1 1 1 -1] (bipolar) and we initialize the weight
matrix (wij) as:
EXAMPLE
weight matrix with no self-connection is:

As per the question, input vector x with missing entries, x = [0 0 1 0] ([x1 x2 x3 x4])
(binary). Make yi = x = [0 0 1 0] ([y1 y2 y3 y4]).
Choosing unit yi (order doesn’t matter) for updating its activation.
Take the ith column of the weight matrix for calculation.
EXAMPLE
In the next steps, for all values of yi, check if there is convergence or not…

Now for next unit, we will take updated value via feedback. (i.e. y = [1 0 1 0])
EXAMPLE

Now for next unit, we will take updated value via feedback. (i.e. y = [1 0 1 0])
EXAMPLE
EXAMPLE
STOCHASTIC NETWORKS
❑A stochastic neural network is a type of artificial neural network that incorporates
randomness in its operations.
❑In stochastic neural networks, the algorithm instead of providing deterministic values
to each neurons it assigns probabilities to each neuron.
❑If each neuron passes the threshold values then only the neurons will fire.
❑It is built by introducing random variation into the network and by giving stochastic
weights.
❑This randomness can enhance learning efficiency, robustness, or enable the network
to model uncertainty.
EXAMPLES OF STOCHASTIC
NEURAL NETWORKS
Boltzmann Machines: A generative stochastic network using energy-based
models.
Dropout Regularization: During training, random neurons are "dropped"
(set to zero) to prevent overfitting.
Bayesian Neural Networks (BNNs): Extend neural networks to include
probability distributions for weights and biases, capturing epistemic
uncertainty.
Stochastic Gradient Descent (SGD):
While not a neural network type, it incorporates randomness in optimization, underpinning
many training processes.
Stochastic Gradient Descent (SGD) is a variant of the Gradient Descent algorithm that is
used for optimizing machine learning models.
EXAMPLE
A stochastic neural network has:
Input Layer: 2 neurons with inputs 𝑥1=3.0 and 𝑥2=1.0
Hidden Layer: 3 neurons, using stochastic activations based on the sigmoid function. For
each hidden neuron:
𝑃(active)=𝜎(𝑤1⋅𝑥1+𝑤2⋅𝑥2+𝑏)
where 𝑤1,𝑤2 are weights, and 𝑏 is the bias.
Assume the following:
Hidden Neuron 1: 𝑤1=0.4,𝑤2=0.6,𝑏=−0.2
Hidden Neuron 2: 𝑤1=−0.3,𝑤2=0.8,𝑏=0.1
Hidden Neuron 3: 𝑤1=0.5,𝑤2=−0.4,𝑏=0.3
Output Layer: 1 neuron that computes:
where 𝑣𝑖 are weights of connections from hidden to output layer:𝑣1=1.0,𝑣2=−1.5,𝑣3=2.0
EXAMPLE
Ques(1): Compute the activation probabilities of the hidden neurons.
Ques(2): Compute the output at the output neuron.
Solution:
Step 1: Calculate activations for the hidden layer neurons
For each hidden neuron:
𝑧=𝑤1⋅𝑥1+𝑤2⋅𝑥2+𝑏
Hidden Neuron 1: 𝑧1=(0.4*3.0)+(0.6*1.0)−0.2 ,
=1.2+0.6−0.2=1.6
EXAMPLE
Hidden Neuron 2:
𝑧2=(−0.3*3.0)+(0.8*1.0)+0.1=−0.9+0.8+0.1=0.0

Hidden Neuron 3:
z3=(0.5*3.0)+(−0.4*1.0)+0.3=1.5−0.4+0.3=1.4

Step 2: Compute the output


The output neuron computes:
EXAMPLE
Substituting the values:
𝑦=(0.832*1.0)+(0.5*−1.5)+(0.802*2.0)
y=0.832−0.75+1.604=1.686
BOLTZMANN MACHINE FOR
PATTERN ENVIRONMENT
STORAGE
The Boltzmann Machine has been introduced as a means to perform global
optimization for multimodal objective functions using the principles of simulated
annealing.
Boltzmann learning is a stochastic learning algorithm. It is named in honor of Ludwig
Boltzmann.
A neural network based on Boltzmann learning is called as a Boltzmann machine.
The neural network consists of input and output layers of neurons with multiple
hidden layers.
The neurons involved operate in a binary manner being either in ON state denoted by
+1 or OFF state denoted by -1.
It operates by randomly choosing a neuron and flipping its state.
ENERGY FUNCTION OF
BOLTZMANN MACHINE
BMs are energy-based models where the energy function determines the joint probability
distribution of the variables.
The energy of Boltzmann machine is defined by,

From the energy, the Boltzmann machine defines the probability distribution over binary patterns.
The lower the energy of a state, the higher its probability.

Where the summation with respect to x is over all of the possible N bit binary values. Namely, the
higher the energy of a pattern x, the less likely that the x is generated.
Applications of Artificial
THANK YOU Intelligence and Neural
Networks(CS-538)

You might also like