Deep Learning

[ QB ,UT-1 ]

1. Explain working of MP model using AND & OR gate ? (Refer from

Book)
Ans - Multilayer Perceptron (MLP) for AND and OR Gates
Multilayer Perceptrons (MLPs) are a type of artificial neural network that
can learn complex patterns. We'll use the simple examples of AND and OR
gates to understand how MLPs work.
AND Gate
 Inputs: Two binary inputs (0 or 1).
 Output: 1 only when both inputs are 1. Otherwise, it's 0.
MLP for AND:
 Hidden Layer: One neuron.
 Output Layer: One neuron.
 Weights and Bias: The key to the MLP's functionality.
 Training: The network is trained on examples of input-output pairs
for the AND gate.
How it works:
1. Inputs: The input values (0 or 1) are fed into the hidden neuron.
2. Weighted Sum: The inputs are multiplied by their respective
weights, and the results are summed.
3. Activation Function: A non-linear activation function (like sigmoid)
is applied to the weighted sum.
4. Output: The output of the hidden neuron is fed to the output
neuron, which also applies an activation function to produce the
final output.
Training: The MLP's weights and bias are adjusted during training to
minimize the difference between the predicted output and the actual
output for each input.
OR Gate
 Inputs: Two binary inputs (0 or 1).
 Output: 1 if at least one input is 1. Otherwise, it's 0.
MLP for OR:
 Structure: Similar to the AND gate MLP.
 Training: The network is trained on examples of input-output pairs
for the OR gate.
Key difference: The weights and bias will be different for the OR gate
compared to the AND gate, reflecting the different logic function.
In essence, MLPs can learn to represent complex patterns like AND and
OR gates by adjusting their weights and biases through training. This
makes them versatile and powerful tools for a wide range of applications.

2. What is need of bias and weight in Neural networks?

Ans - Bias and weights are crucial components of neural networks.
 Bias: Bias refers to the error due to overly simplistic assumptions in
the learning algorithm. Acts as a threshold, determining the
activation of a neuron. It allows the network to learn patterns that
are not linearly separable.
 Weights: Represent the strength of the connections between
neurons. They determine how much influence each input has on
the output of a neuron.
By adjusting these parameters during training, neural networks can learn
complex patterns and make accurate predictions.
3. Why we need update bias and weight in neural network?
Ans - Updating bias and weights is essential for neural networks to learn
and adapt.
 Learning: Neural networks learn by adjusting their parameters to
minimize the error between their predictions and the actual
outcomes.
 Adaptation: By updating bias and weights, the network can adapt to
new data and improve its performance over time.
 Flexibility: This flexibility allows neural networks to handle complex
tasks and patterns that are not easily represented by simple models.
In essence, updating bias and weights is the core mechanism through
which neural networks learn and improve.

4. Explain how perceptron model differ from MP Model?

Ans - Perceptron and McCulloch-Pitts (MP) Model are both early
computational models of artificial neurons, but they have some key
differences:
 Inputs: MP models only accept binary inputs (0 or 1), while
perceptrons can handle real-valued inputs.
 Weights: MP models have fixed weights, while perceptrons can
adjust their weights during training to learn patterns.
 Activation Function: MP models use a simple threshold function,
while perceptrons can use more complex activation functions like
sigmoid or ReLU.
 Learning: Perceptrons have a built-in learning rule that allows them
to adjust their weights to improve their performance on a given
task.
In essence, perceptrons are a more flexible and powerful extension of the
MP model, incorporating features that enable them to learn and adapt to
a wider range of tasks.
5. Explain perceptron learning algorithm?
Ans – Frank Rosenblatt an American psychologist proposed the classic
Perceptron model in 1958 .Perceptron learning is a supervised learning
algorithm used to train single-layer neural networks. It works by adjusting
the weights and bias of the perceptron based on the errors it makes in
classifying training data. Here's how it works:
1. Initialize weights and bias: Start with random values for the weights
and bias.
2. Present training data: Feed a training example to the perceptron.
3. Calculate output: The perceptron computes its output using the
weighted sum of inputs and the bias, followed by an activation
function.
4. Calculate error: Compare the predicted output with the actual
target output.
5. Update weights and bias: If the predicted output is incorrect, adjust
the weights and bias to move the decision boundary closer to the
correct classification.
6. Repeat (Epoch): Repeat steps 2-5 for all training examples until the
perceptron converges or reaches a desired level of accuracy.
Perceptron learning is a simple but effective algorithm for training single-
layer neural networks, particularly for linearly separable problems.
However, it may not converge for non-linearly separable data.
6. Difference between linearly separable v/s linearly non-separable
classes?
Ans - Linearly separable classes can be divided by a straight line or
hyperplane in a two-dimensional or higher-dimensional space. This means
that there is a clear boundary between the classes, and they can be easily
separated using a linear decision rule.
Linearly non-separable classes cannot be divided by a straight line or
hyperplane. This means that there is no simple linear decision boundary
that can separate the classes, and more complex models or techniques
are required to classify them accurately.
In essence, linearly separable classes can be easily classified using a
simple linear model, while linearly non-separable classes require more
complex models to capture the underlying patterns.

7. Construct 2 input AND gate using Perceptron Model?

Ans - Constructing an AND Gate Using a Perceptron
Understanding the AND Gate: An AND gate is a logic gate that outputs 1
only when both inputs are 1. Otherwise, it outputs 0.
Perceptron Model for AND Gate: A perceptron can be used to implement
the AND gate. A perceptron is a simple artificial neuron that takes a
weighted sum of its inputs and applies an activation function to produce
an output. Steps:
1. Define the perceptron:
o Inputs: Two inputs, x1 and x2.
o Weights: Two weights, w1 and w2.
o Bias: One bias, b.
o Activation function: Step function (outputs 1 if the weighted
sum is greater than or equal to a threshold; otherwise, it
outputs 0).
 2. Set initial values:
o Weights: Choose initial random values for w1 and w2.
o Bias: Choose an initial random value for b.
3. Training:
o Input data: Use the following training data:
 (0, 0) -> 0
 (0, 1) -> 0
 (1, 0) -> 0
 (1, 1) -> 1
o Update weights and bias: For each training example, calculate
the predicted output using the perceptron's equation:
 predicted_output = step(w1x1 + w2x2 + b)
 If the predicted output is incorrect, update the weights
and bias using the perceptron learning rule:
 w1 = w1 + learning_rate * (target_output -
predicted_output) * x1
 w2 = w2 + learning_rate * (target_output -
predicted_output) * x2
 b = b + learning_rate * (target_output -
predicted_output)
 Where learning_rate is a hyperparameter that controls
the rate of learning.
4. Repeat: Repeat the training process until the perceptron correctly
classifies all training examples.
Example:
 Choose initial values: w1 = 0.5, w2 = 0.5, b = -0.7.
 Train the perceptron using the training data.
  After training, the perceptron's weights and bias may converge to
values like w1 = 0.5, w2 = 0.5, and b = -0.7.
 With these values, the perceptron will correctly implement the AND
gate.
By following these steps, you can construct a perceptron model that
effectively implements the AND gate.

8. Construct 2 input OR gate using Perceptron Model?

Ans - Constructing an OR Gate Using a Perceptron
Understanding the OR Gate: An OR gate is a logic gate that outputs 1 if at
least one input is 1. Otherwise, it outputs 0.
Perceptron Model for OR Gate: Similar to the AND gate, a perceptron can
be used to implement the OR gate.
Steps:
1. Define the perceptron:
o Inputs: Two inputs, x1 and x2.
o Weights: Two weights, w1 and w2.
o Bias: One bias, b.
o Activation function: Step function.
2. Set initial values:
o Choose random initial values for weights and bias.
 3. Training:
o Use training data: (0, 0) -> 0, (0, 1) -> 1, (1, 0) -> 1, (1, 1) -> 1.
o Update weights and bias using the perceptron learning rule as
explained in the AND gate construction.
4. Repeat: Train until the perceptron correctly classifies all training
examples.
Key difference from AND gate:
 The optimal weights and bias for the OR gate will likely be different
from the AND gate due to the different logic function.
Note: The perceptron learning rule is a simple algorithm that works well
for linearly separable problems like AND and OR gates. For more complex
problems, more advanced techniques like backpropagation may be
necessary.
9. Explain working of Multilayer Perceptron model?
Ans - Multilayer Perceptron (MLP): A Neural Network for Complex
Patterns
A Multilayer Perceptron (MLP) is a type of artificial neural network that
can learn complex patterns. Unlike a single-layer perceptron, which can
only classify linearly separable data, an MLP can handle more complex
tasks.
How it works:
1. Input Layer: Receives the input data.
2. Hidden Layers: These layers process the input data and extract
features. Each neuron in a hidden layer takes a weighted sum of its
inputs, applies an activation function (like sigmoid or ReLU), and
passes the result to the next layer.
3. Output Layer: Produces the final output, usually a classification or
prediction.
Training:
 Backpropagation: A common training algorithm that calculates the
error between the predicted output and the actual target. It then
adjusts the weights and biases of the network to minimize the error.
 Optimization: Techniques like gradient descent are used to find the
optimal weights and biases.
In essence, MLPs are powerful tools that can learn from data and make
accurate predictions on new, unseen data. They have been applied
successfully in various fields, including image recognition, natural
language processing, and finance.
10. Explain delta learning rule?
Ans - The delta learning rule is a fundamental algorithm also known as
error correction rule. The primary goal of the Delta learning rule is to
minimize the error between the desired output (target value) and the
actual output produced by a neural network.
How it works:
1. Calculate error: Determine the difference between the desired
output and the actual output of a neuron.
2. Adjust weights: Update the weights of the connections to the
neuron in a way that reduces the error.
3. Repeat: Iterate this process for all neurons in the network until the
desired level of accuracy is achieved.
Key points:
 Error signal: The error signal is used to guide the weight updates.
 Gradient descent: The delta learning rule is often implemented
using gradient descent, which involves moving in the direction of
steepest descent to minimize the error.
 Generalization: The delta learning rule can be applied to various
types of neural networks, including perceptrons and multilayer
perceptrons.
In essence, the delta learning rule is a powerful tool for training neural
networks, providing a mechanism for them to learn from their mistakes
and improve their performance over time.

11. Numerical to find Output of single neuron ( Refer class notes for
example)
12. Numerical to find Output of Neural Network ( ( Refer class notes for
example )
13. Different types of Activation function?
Ans - Activation Functions in Neural Networks
Activation functions introduce non-linearity into neural networks,
allowing them to learn complex patterns. Here are some common types:
 Sigmoid: Returns values between 0 and 1, often used in output
layers for classification tasks.
 ReLU (Rectified Linear Unit): Returns the input if it's positive,
otherwise returns 0. It's popular due to its computational efficiency
and ability to avoid the vanishing gradient problem.
 Tanh: Returns values between -1 and 1, similar to sigmoid but with
a wider range.
 Leaky ReLU: A variant of ReLU that allows for a small slope for
negative inputs, helping to avoid the dying ReLU problem.
 Softmax: Often used in the output layer for multi-class
classification, ensuring that the outputs sum to 1 and represent
probabilities.
The choice of activation function depends on the specific task and
network architecture.

14. What you mean by Deep Network?

Ans - Deep Networks are neural networks with multiple hidden layers.
This depth allows them to learn complex patterns and features in data
that shallower networks cannot. They are particularly effective for tasks
such as image recognition, natural language processing, and speech
recognition.
The term "deep" refers to the number of layers, not necessarily the size of
the network. A deep network can have relatively few neurons per layer,
but still be effective if the layers are well-structured and trained
appropriately.
15. What are components of Deep Learning?
Ans - Deep Learning Components
Deep learning models are composed of several key components:
1. Neurons: The fundamental building blocks of neural networks. Each
neuron receives inputs, processes them, and produces an output.
2. Layers: Groups of neurons organized into layers. There are typically
three types of layers:
o Input layer: Receives the raw data.
o Hidden layers: Process the data and extract features.
o Output layer: Produces the final output, such as a
classification or prediction.
3. Weights and biases: Parameters that determine the strength of
connections between neurons and the activation threshold.
4. Activation functions: Introduce non-linearity into the network,
enabling it to learn complex patterns.
5. Loss function: Measures the difference between the predicted
output and the actual target.
6. Optimizer: An algorithm used to update the weights and biases to
minimize the loss function.
These components work together to create powerful models that can
learn complex patterns from data.
16. Difference between biological Neuron and Artificial Neuron ?

17. What are challenges in Deep Learning?

Ans- Challenges in Deep Learning
 Data scarcity: Deep learning models require large amounts of high-
quality data to train effectively.
 Overfitting: Models can become too complex and fit the training
data too well, leading to poor performance on new data.
 Interpretability: Deep learning models can be difficult to
understand and interpret, making it challenging to explain their
decisions.
 Computational resources: Training deep learning models can be
computationally expensive, requiring powerful hardware.
 Transfer learning: Applying deep learning models to new domains
can be challenging due to the need for domain-specific knowledge.
 Ethical considerations: The use of deep learning raises ethical
concerns, such as bias, privacy, and accountability.