Scribd
Upload
20 views31 pages

Artificial Neural Network

Artificial Neural Networks (ANNs) are statistical learning models inspired by biological neural networks, consisting of interconnected 'neurons' that facilitate supervised learning. The processing of ANNs relies on network topology, weight adjustments, and activation functions, with training involving forward and back propagation to minimize error. The document details the structure of ANNs, including input, hidden, and output layers, and explains the training process through calculations of net input and error adjustments.

Uploaded by

athakur2be22
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
20 views31 pages

Artificial Neural Network

Artificial Neural Networks (ANNs) are statistical learning models inspired by biological neural networks, consisting of interconnected 'neurons' that facilitate supervised learning. The processing of ANNs relies on network topology, weight adjustments, and activation functions, with training involving forward and back propagation to minimize error. The document details the structure of ANNs, including input, hidden, and output layers, and explains the training process through calculations of net input and error adjustments.

Uploaded by

athakur2be22
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 31

Artificial Neural Network

Dr. Singara Singh Kasana


Associate Professor
Computer Science and Engineering Department
Thapar Institute of Engineering and Technology
Patiala, Punjab
Artificial Neural Networks

 Artificial neural networks are statistical learning models, inspired


by biological neural networks (central nervous systems, such as
the brain) and are used in machine learning.
 These networks are represented as systems of interconnected
“neurons”, which send messages/data to each other.
 The connections within the network can be systematically
adjusted based on inputs and outputs, making them ideal for
supervised learning.
Processing of ANN depends upon the following three building blocks
 Network Topology
 Adjustments of Weights or Learning
 Activation Functions
Network Topology
A network topology is the arrangement of a network along with its
nodes and connecting lines. According to the topology, ANN can be
classified as the following kinds −
Feedforward Network
It is a non-recurrent network having processing units/nodes in layers
and all the nodes in a layer are connected with the nodes of the
previous layers. The connection has different weights upon them.
There is no feedback loop means the signal can only flow in one
direction, from input to output. It may be divided into the following
two types −
Single layer feedforward network − The concept is of feedforward
ANN having only one weighted layer. In other words, we can say the
input layer is fully connected to the output layer.
Multilayer feedforward network − The concept is of feedforward
ANN having more than one weighted layer. As this network has one
or more layers between the input and the output layer, it is called
hidden layers.
A neural network is a collection of “neurons” with “synapses”
connecting them.
The collection is organized into three main parts:
1. input layer,
2. hidden layer(s)
3. output layer.
Note that you can have n hidden layers, with the term “deep”
learning implying multiple hidden layers.
Training a neural network basically means calibrating all of the
“weights” by repeating two key steps, forward propagation and back
propagation.

1. In forward propagation, we apply a set of weights to the input


data and calculate an output. For the first forward propagation,
the set of weights is selected randomly.

2. In back propagation, we measure the margin of error of the


output and adjust the weights accordingly to decrease the error.
For the general model of artificial neural network, the net input can be calculated as follows

−
Neural Network with one input, two hidden neurons, two output neurons
Neural Network with two inputs, two hidden neurons, two output neurons
Additionally, the hidden and output neurons may include a bias.
we’re going to work with a single training set: given inputs 0.05 and 0.10, we want
the neural network to output 0.01 and 0.99.
The Forward Pass
To begin, lets see what the neural network currently predicts given
the weights and biases and inputs of 0.05 and 0.10. To do this we’ll
feed those inputs forward though the network.
We figure out the total net input to each hidden layer
neuron, squash the total net input using an activation function, then
repeat the process with the output layer neurons.
Here’s how we calculate the total net input for hidden node h1

We then squash it using the sigmoid function to get the output of h1

Carrying out the same process for hidden node h2, we get:
We repeat this process for the output layer neurons, using the output from the hidden layer neurons as inputs.

Here’s the output for o1:

And carrying out the same process for o2 we get:


Calculating the Total Error
We can now calculate the error for each output neuron using
the squared error function and sum them to get the total error:

The1/2 is included so that exponent is cancelled when we this


function will be differentiated.
Back Propagation

The objective of the backpropagation is to update each of the

weights in the network so that they cause the predicted output to be

closer to the target output, thereby minimizing the error for each

output neuron and the network as a whole.


Output Layer
Consider w5. We want to know how much a change in w5
affects the total error, aka
… (1)
The partial derivative of the logistic function is the output
multiplied by 1 minus the output:

… (2)

Finally, how much does the total net input of o1 change with respect
to w5?

Taken from forward


propagation
Putting all these together:

You’ll often see this calculation combined in the form of the delta rule:
To decrease the error, we then subtract this value from the current weight (optionally multiplied by some learning
rate, set to 0.5 in this example).

Repeating for w6, w7 and w8


Hidden Layer
Next, we’ll continue the backwards pass by calculating new values
for w1, w2, w3 and w4
We’re going to use a similar process as we did for the output layer, but slightly different to account for the fact that the
output of each hidden layer neuron contributes to the output (and therefore error) of multiple output neurons. We
know that affects both and

Using (1) and (2)

… (3)
… (4)

Using (3) and (4)

Therefore
Now that we have we need to figure out and then for each weight

We calculate the partial derivative of the total net input to h1 with respect to w1 the same as we did for the
output neuron:
Putting it all together:

We can now update w1:


Repeating this for w2, w3, and w4
Finally, we’ve updated all of our weights! When we feed forward the
0.05 and 0.1 inputs originally, the error on the network was
0.298371109.

After first round of backpropagation, the total error is now down to


0.291027924. It might not seem like much, but after repeating this
process 10,000 times, for example, the error plummets to
0.0000351085. At this point, when we feed forward 0.05 and 0.1, the
two outputs neurons generate 0.015912196 (vs 0.01 target) and
0.984065734 (vs 0.99 target).
Thanks

You might also like