UNIT 1 and 2 :

Q) define the neuron and model of neuron ?

Neurons (also called neurones or nerve cells) are the fundamental units of the brain and nervous
system, the cells responsible for receiving sensory input from the external world, for sending motor
commands to our muscles, and for transforming and relaying the electrical signals at every step in
between. More than that, their interactions define who we are as people. Having said that, our roughly
100 billion neurons do interact closely with other cell types, broadly classified as glia (these may
actually outnumber neurons, although it’s not really known).

Q) Expalin the diff activation function in details ?

Step Function:
Step Function is one of the simplest kind of activation functions. In this, we
consider a threshold value and if the value of net input say y is greater than the
threshold then the neuron is activated.
Mathematically,

Given below is the graphical representation of step function.

Here are some of the limitations of binary step function:

 It cannot provide multi-value outputs—for example, it cannot be used for

multi-class classification problems.
 The gradient of the step function is zero, which causes a hindrance in the
backpropagation process.

Sigmoid Function:
Sigmoid function is a widely used activation function. It is defined as:
This is a smooth function and is continuously differentiable. The biggest
advantage that it has over step and linear function is that it is non-linear. This is
an incredibly cool feature of the sigmoid function. This essentially means that
when I have multiple neurons having sigmoid function as their activation
function – t
he output is non linear as well. The function ranges from 0-1 having an S shape.

 Binary sigmoidal function − This activation function performs

input editing between 0 and 1. It is positive in nature. It is

 always bounded, which means its output cannot be less than
0 and more than 1. It is also strictly increasing in nature,
which means more the input higher would be the output. It
can be defined as
F(x)=sigm(x)=11+exp(−x)
  ipolar sigmoidal function − This activation function performs *-
+input editing between -1 and 1. It can be positive or
negative in nature. It is always bounded, which means its
output cannot be less than -1 and more than 1. It is also
strictly increasing in nature like sigmoid function. It can be
defined as
F(x)=sigm(x)=21+exp(−x)−1=1−exp(x)1+exp(x)

Here’s why sigmoid/logistic activation function is one of the most widely used
functions:
 It is commonly used for models where we have to predict the probability
as an output. Since probability of anything exists only between the range
of 0 and 1, sigmoid is the right choice because of its range.
 The function is differentiable and provides a smooth gradient, i.e.,
preventing jumps in output values. This is represented by an S-shape of
the sigmoid activation function.

Linear Activation Function

 The linear activation function, also known as "no activation," or "identity

function" (multiplied x1.0), is where the activation is proportional to the
input.
 The function doesn't do anything to the weighted sum of the input, it
simply spits out the value it was given.

However, a linear activation function has two major problems :

 It’s not possible to use backpropagation as the derivative of the function
is a constant and has no relation to the input x.
 All layers of the neural network will collapse into one if a linear activation
function is used. No matter the number of layers in the neural network,
 the last layer will still be a linear function of the first layer. So, essentially,
a linear activation function turns the neural network into just one layer.

Q) Define Bias and Threshold?

Bias in Neural Networks
Definition: In neural networks, bias is an additional parameter (bias term or
bias unit) associated with each neuron in a layer, except for the input layer. It
allows the neural network to model more complex relationships by providing
an offset to the weighted sum of inputs.
Purpose: The bias term allows the activation function of a neuron to be shifted,
influencing when the neuron should activate or deactivate based on the input.
It provides flexibility in the modeling process.
Mathematically: If �1,�2,…,��x1,x2,…,xn are the inputs to a neuron, �1,�2,
…,��w1,w2,…,wn are the corresponding weights, and �b is the bias, then the
weighted sum is �=�1�1+�2�2+…+����+�z=w1x1+w2x2+…+wnxn+b.
The output of the neuron after applying the activation function �(�)σ(z) is influenced by
this bias term.

Threshold in Neural Networks:

Definition: In neural networks, the term "threshold" is not used in the same
way as in traditional binary classification. Instead, the concept of a threshold is
embedded in the activation function. The activation function determines
whether a neuron should be activated or not based on the input it receives.

Purpose: The activation function introduces non-linearity to the neural

network, allowing it to learn and represent complex patterns in data. The
activation function's threshold-like behavior determines when a neuron should
"fire" or be activated.

Example: In a step function activation (commonly used in perceptrons), the

neuron activates if the weighted sum of inputs exceeds a certain threshold, and
it remains inactive otherwise. However, more commonly used activation
functions like sigmoid, tanh, or ReLU have a smoother transition between
activation and deactivation without a clear threshold value.

Q) State the training algoritham used for the Hebb Network ?

Hebbian Learning Rule, also known as Hebb Learning Rule, was proposed by
Donald O Hebb. It is one of the first and also easiest learning rules in the neural
network. It is used for pattern classification. It is a single layer neural network,
i.e. it has one input layer and one output layer. The input layer can have many
units, say n. The output layer only has one unit. Hebbian rule works by
updating the weights between neurons in the neural network for each training
sample.

Hebbian Learning Rule Algorithm :

1. Set all weights to zero, wi = 0 for i=1 to n, and bias to zero.

2. For each input vector, S(input vector) : t(target output pair), repeat steps
3-5.
3. Set activations for input units with the input vector Xi = Si for i = 1 to n.
4. Set the corresponding output value to the output neuron, i.e. y = t.
5. Update weight and bias by applying Hebb rule for all i = 1 to n:

Q) State the characteristic of ANN ?

Artificial Neural Networks (ANNs) have several key characteristics that define their
structure, function, and application. Here are some of the main characteristics of
Artificial Neural Networks:

1. Neurons (Nodes):
 ANNs consist of interconnected nodes or artificial neurons that mimic
the structure and function of biological neurons.
  Neurons receive input, apply a weighted sum, and pass the result
through an activation function to produce an output.
2. Layers:
 ANNs are organized into layers, including an input layer, one or more
hidden layers, and an output layer.
 The input layer receives external data, hidden layers process
information, and the output layer produces the final result.
3. Weights and Connections:
 Connections between neurons are represented by weights, which
determine the strength of the connection.
 During training, weights are adjusted to learn patterns and
relationships in the data.
4. Activation Functions:
 Neurons use activation functions to introduce non-linearity into the
network.
 Common activation functions include sigmoid, hyperbolic tangent
(tanh), and rectified linear unit (ReLU).
5. Learning Algorithms:
 ANNs use learning algorithms to adjust weights and learn from data
during a training phase.
 Backpropagation is a widely used algorithm for supervised learning in
ANNs.
6. Training and Learning:
 ANNs learn from data through a training process where input-output
pairs are used to adjust weights.
 Learning involves minimizing a cost or error function, typically through
gradient descent or a variant.
7. Universal Approximators:
 ANNs are known as universal function approximators, meaning they
can theoretically learn and represent any complex function given
sufficient neurons and layers.
8. Parallel Processing:
 ANNs can perform parallel processing, with multiple neurons in a layer
processing information simultaneously.
 This parallelism contributes to the network's ability to handle large
amounts of data.
9. Adaptability:
 ANNs are adaptive and can generalize patterns from training data to
make predictions or classifications on new, unseen data.
 They can adapt to changes in the input patterns over time.
10. Robustness:
  ANNs can exhibit robustness in the presence of noise or incomplete
information, allowing them to handle imperfect or ambiguous data.
11. Applications:
 ANNs are applied in various fields, including image and speech
recognition, natural language processing, pattern recognition, and
control systems.
12. Deep Learning:
 Deep learning, a subset of machine learning, involves deep neural
networks with multiple hidden layers, enabling the modeling of
complex hierarchical representations.

Q)Diff bet supervised learning and unsupervised learning ?

Supervised Learning Unsupervised Learning

Supervised learning algorithms are trained Unsupervised learning algorithms are

using labeled data. trained using unlabeled data.

Supervised learning model takes direct Unsupervised learning model does not
feedback to check if it is predicting correct take any feedback.
output or not.

Supervised learning model predicts the Unsupervised learning model finds the
output. hidden patterns in data.

In supervised learning, input data is In unsupervised learning, only input data

provided to the model along with the is provided to the model.
output.

The goal of supervised learning is to train The goal of unsupervised learning is to

the model so that it can predict the output find the hidden patterns and useful
when it is given new data. insights from the unknown dataset.

Supervised learning needs supervision to Unsupervised learning does not need

train the model. any supervision to train the model.

Supervised learning can be categorized Unsupervised Learning can be classified

in Classification and Regression problem in Clustering and Associations problem
s. s.

Supervised learning can be used for those Unsupervised learning can be used for
cases where we know the input as well as those cases where we have only input
 corresponding outputs. data and no corresponding output data.

Q)Implement AND function using McCulochs neuron network ?

https://youtu.be/e33eO0yJ5HE

Q)8
Q) 9

Q)10 DRAW AND explain Hebb network in details ?

Hebbian Learning Rule, also known as Hebb Learning Rule, was proposed by
Donald O Hebb.
It is one of the first and also easiest learning rules in the neural network. It is
used for pattern
classification.
It is a single layer neural network, i.e. it has one input layer and one output
layer. The input layer can
have many units, say n.
The output layer only has one unit. Hebbian rule works by updating the
weights between neurons in the
neural network for each training sample.
The weight and bias update in Hebb rule is given by:

wi(new) = wi (old) + x;y

b(new) = b(old) +y

Use example __________

Q) Draw the diagram of the biological neural network ?

Q) Diff between artificial neural network and biological neural network ?

Parameters ANN BNN

input dendrites
weight synapse
Structure
output axon
hidden layer cell body

very precise structures and

Learning they can tolerate ambiguity
formatted data

complex simple

Processor high speed low speed

one or a few large number

separate from a processor integrated into processor

Memory localized distributed

non-content addressable content-addressable
 centralized distributed

Computing sequential parallel

stored programs self-learning

Reliability very vulnerable robust

numerical and symbolic perceptual

Expertise
manipulations problems

well-defined poorly defined

Operating
Environment well-constrained un-constrained

the potential of fault performance degraded even

Fault Tolerance
tolerance on partial damage

Q) explain the perceptron network with suitable diagram ?

Perceptron is Machine Learning algorithm for supervised learning of various binary

classification tasks. Further, Perceptron is also understood as an Artificial Neuron
or neural network unit that helps to detect certain input data computations in
business intelligence.

Perceptron model is also treated as one of the best and simplest types of Artificial
Neural networks. However, it is a supervised learning algorithm of binary classifiers.
Hence, we can consider it as a single-layer neural network with four main parameters,
i.e., input values, weights and Bias, net sum, and an activation function.
Basic Components of Perceptron

o Input Nodes or Input Layer:

This is the primary component of Perceptron which accepts the initial data into the
system for further processing. Each input node contains a real numerical value.

o Wight and Bias:

Weight parameter represents the strength of the connection between units. This is
another most important parameter of Perceptron components. Weight is directly
proportional to the strength of the associated input neuron in deciding the output.
Further, Bias can be considered as the line of intercept in a linear equation.

o Activation Function:

These are the final and important components that help to determine whether the
neuron will fire or not. Activation Function can be considered primarily as a step
function.

Types of Activation functions:

o Sign function
o Step function, and
o Sigmoid function
The data scientist uses the activation function to take a subjective decision based on
various problem statements and forms the desired outputs. Activation function may
differ (e.g., Sign, Step, and Sigmoid) in perceptron models by checking whether the
learning process is slow or has vanishing or exploding gradients.

Characteristics of Perceptron
The perceptron model has the following characteristics.

1. Perceptron is a machine learning algorithm for supervised learning of binary

classifiers.
2. In Perceptron, the weight coefficient is automatically learned.
3. Initially, weights are multiplied with input features, and the decision is made whether
the neuron is fired or not.
4. The activation function applies a step rule to check whether the weight function is
greater than zero.
5. The linear decision boundary is drawn, enabling the distinction between the two
linearly separable classes +1 and -1.
6. If the added sum of all input values is more than the threshold value, it must have an
output signal; otherwise, no output will be shown.

Q) With the neat flowchart explain the perceptron training algoritham for
single input ?
Q) state the testing algoritham used In perc eptron ?

Perceptron Network Testing Algorithm

The testing algorithm is asfollows:

Step 0: The initialweights to be used here are taken from the training algorithms.

Step 1: For each input vector X to be classified, perform Steps 2-3.

Step 2: Set activations of the input unit.

Step 3: Obtain the response of output unit

Q) Explain the Adaline network with suitable diagram ?

Adaptive Linear Neuron (Adaline)
The units with linear activation function are called linear units. A network with
a single linear unit is calledan Adaline (adaptive linear neuron). That is, in an
Adaline, the input-output relationship is linear. Adalineusesbipolar activation
for its input signals and its target output. The weights between the input and
theoutput are adjustable. The bias in Adaline acts like an adjustable weight,
whose connection is from a unitwith activations being always 1. Adaline is a net
which has only one output unit. The Adaline network maybe trained using
delta rule. The delta rule may also be called as least mean square (LMS) rule or
Widrow-Hoffrule. This learning rule is found to minimize the meansquared
error between the activation and the targetvalue.
Q) construct a flowchart for the Adaline training algoritham for single output ?

Q) Explain the testing algoritham used in Adaline network ?

 x.

Q) explain the back prapogation network with suitable diagram ?

The backpropagation learning algorithm is one of the most important
developments in neural networks. The networks associatedwith back-
propagation learning algorithm are called back propagation networks(BPNs).
The basic concept for this weight update algorithm is simply thegradient-
descent method. This is amethod where the error is propagated back to the
hidden unit. The aim of the neural networkistotrain thenet to achieve a
balance between the net's ability to respond and its ability to give
reasonableresponses to the inputthat is similar but not identical to the one
that is used in training. The back-propagation algorithm is different from other
networks in respect to the process by which weights are calculated during the
learning period of the network
A back-propagation neural network is a multilayer, feed forward neural network
consisting of an input layer,a hidden layer and an output layer. The neurons
present in the hidden and output layers have biases, whichare the connections
from the units whose activation is always 1. The bias terms also acts as
weights. The Figure below shows the architecture of a BPN, depicting only the
direction of information flow for the feed-forward phase.During the back
propagation phase of learning signals are sent in the reverse direction. The
inputs sent to the BPN and the output obtained from the net could be either
binary (0, 1) orbipolar ( -1, + 1). The activation function could be any function
which increases monotonically and is alsodifferentiable.

Q) construct a flowchart for back prapogation training algoritham for single

output ?
Flowchart for Training Process The flowchart for the training process using a BPN is shown in Figure
below. The terminologies used in theflowchart and in the training algorithm are as follows: