NEURAL NETWORK

Prof. Sushil Kumar Azad

Neural Networks is a computational learning system that uses a network of functions to
understand and translate a data input of one form into a desired output, usually in another form.
The concept of the artificial neural network was inspired by human biology and the way neurons
of the human brain function together to understand inputs from human senses

In simple words, Neural Networks are a set of algorithms that tries to recognize the patterns,
relationships, and information from the data through the process which is inspired by and
works like the human brain/biology.
Components / Architecture of Neural Network A simple neural network consists of three components :

• Input layer
• Hidden layer
• Output layer
Input Layer: Also known as Input nodes are the inputs/information from the outside world is
provided to the model to learn and derive conclusions from. Input nodes pass the information
to the next layer i.e Hidden layer.

Hidden Layer: Hidden layer is the set of neurons where all the computations are performed on
the input data. There can be any number of hidden layers in a neural network. The simplest
network consists of a single hidden layer.

Output layer: The output layer is the output/conclusions of the model derived from all the
computations performed. There can be single or multiple nodes in the output layer. If we have
a binary classification problem the output node is 1 but in the case of multi-class classification,
the output nodes can be more than 1.
The typical Artificial Neural Network looks something like the above figure.
Relationship between Biological neural network and artificial neural network:

Biological Neural Network Artificial Neural Network (ANN)

Dendrites Inputs

Cell nucleus Nodes

Synapse Weights

Axon Output
 The artificial neural network takes input and computes the
weighted sum of the inputs and includes a bias. This computation is
represented in the form of a transfer function.

It determines weighted total is passed as an input to an

activation function to produce the output. Activation functions choose
whether a node should fire or not. Only those who are fired make it to
the output layer. There are distinctive activation functions available that
can be applied upon thesort of task we are performing.
Perceptron and Multi-Layer Perceptron

Perceptron is a simple form of Neural Network and consists

of a single layer where all the mathematical computations are

performed .

Whereas, Multilayer Perceptron also known as Artificial

Neural Networks consists of more than one perception which is

grouped together to form a multiple layer neural network.

In the above figure, The Artificial Neural Network consists of four

layers interconnected with each other:

 An input layer, with 6 input nodes

 Hidden Layer 1, with 4 hidden nodes/4 perceptrons
 Hidden layer 2, with 4 hidden nodes
 Output layer with 1 output node
Step by Step Working of the Artificial Neural Network
1. In the first step, Input units are passed i.e data is passed with

some weights attached to it to the hidden layer. We can have

any number of hidden layers. In the above image inputs

x1,x2,x3,….xn is passed.
2. Each hidden layer consists of neurons. All the inputs are connected to
each neuron.

3. After passing on the inputs, all the computation is performed in the

hidden layer.
Computation performed in hid d en layers are d one in two steps which are as
follows :

 First of all, all the inputs are multiplied by their weights.

Weight is the grad ient or coefficient of each variable. It shows

the strength of the particular input. After assigning the weights,

a bias variable is ad d ed . Bias is a constant that helps the mod el

to fit in the best waypossible.

Z1 = W1 *In1 + W2 *In2 + W3 *In3 + W4 *In4 + W5 *In5 + b

W1 , W2 , W3 , W4 , W5 are the weights assigned to the inputs In1 , In2 , In3 , In4, In5,
and b is the bias.

 Then in the second step, the activation function is applied to

the linear equation Z1. The activation function is a non-linea r
transformation that is applied to the input before send ing it to the
next layer of neurons. The importance of the activation function
is to inculcate nonlinearity in the mod el.
4. The whole process described in point 3 is performed in each hidden layer.
After passing through every hidden layer, we move to the last layer i.e our output
layer which gives us the final output.

The process explained above is known as forwarding Propagation.

5. After getting the predictions from the output layer, the error is calculated
i.e the difference between the actual and the predicted output.

If the error is large, then the steps are taken to minimize the error and for the same
purpose, Back Propagation is performed.
What is Back Propagation and How it works?
Back Propagation is the process of updating and finding the optimal values of weights
or coefficients which helps the model to minimize the error i.e difference between the
actual and predicted values.

The weights are updated with the help of optimizers. Optimizers are the methods/
mathematical formulations to change the attributes of neural networks i.e weights to
minimize the error.
Activation Functions

Activation functions are attached to each neuron and are

mathematical equations that determine whether a neuron should be

activated or not based on whether the neuron’s input is relevant for the

model’s prediction or not. The purpose of the activation function is to

introduce thenonlinearity in the data.

Various Types of Activation Functions are :

 Sigmoid Activation Function

 TanH / Hyperbolic Tangent Activation Function
 Rectified Linear Unit Function (ReLU)
 Leaky ReLU 
 Softmax
What Is a Convolutional Neural Network?
A convolutional neural network is one adapted for analysing and
identifying visual data such as digital images or photographs.

What Is a Recurrent Neural Network?

A recurrent neural network is one adapted for analysing time series
data, event history, or temporal ordering.

What Is a Deep Neural Network?

Also known as a deep learning network, is one that involves two or more
processing layers.
Types of Artificial Neural Network:

There are various types of Artificial Neural Networks (ANN) depending upon the human brain neuron and
network functions, an artificial neural network similarly performs tasks. For example, segmentation or
classification.

• Feedback ANN:

The feedback networks feed information back into itself and are well suited to solve optimization issues.
The Internal system error corrections utilize feedback ANNs.

• Feed-Forward ANN:

A feed-forward network is a basic neural network comprising of an input layer, an output layer, and at least
one layer of a neuron. Through assessment of its output by reviewing its input, the intensity of the network
can be noticed based on group behaviour of the associated neurons, and the output is decided. The
primary advantage of this network is that it figures out how to evaluate and recognize input patterns.
Advantages of Artificial Neural Network

• Parallel processing capability

• Storing data on the entire network
• Capability to work with incomplete knowledge
• Having a memory distribution
• Having fault tolerance

Disadvantages of Artificial Neural Network

• Assurance of proper network structure
• Unrecognized behaviour of the network
• Hardware dependence
• Difficulty of showing the issue to the network
• The duration of the network is unknown
Importance of Neural Network:
o Without Neural Network:
Let's have a look at the example given below. Here
we have a machine, such that we have trained it
with four types of cats, as you can see in the image
below. And once we are done with the training, we
will provide a random image to that particular
machine that has a cat. Since this cat is not similar
to the cats through which we have trained our
system, so without the neural network, our
machine would not identify the cat in the picture.
Basically, the machine will get confused in figuring
out where the cat is.
o With Neural Network:

However, when we talk about the case with a neural network,

even if we have not trained our machine with that particular
cat. But still, it can identify certain features of a cat that we
have trained on, and it can match those features with the cat
that is there in that particular image and can also identify the
cat.