Q1a) What is a neural network activation function? State its types.

Answer:

1. Definition:

o An activation function is a mathematical function applied to a neuron’s

output to introduce non-linearity.

o It determines whether a neuron should be activated based on weighted

inputs.

2. Purpose:

o Helps neural networks learn complex patterns.

o Allows backpropagation-based learning.

3. Types of Activation Functions:

o Linear Activation Function: f(x)=axf(x) = axf(x)=ax

o Non-Linear Activation Functions:

 Sigmoid Function: f(x)=11+e−xf(x) = \frac{1}{1+e^{-x}}f(x)=1+e−x1

 Tanh Function: f(x)=ex−e−xex+e−xf(x) = \frac{e^x - e^{-x}}{e^x + e^{-

x}}f(x)=ex+e−xex−e−x

 ReLU (Rectified Linear Unit): f(x)=max(0,x)f(x) = max(0, x)f(x)=max(0,x)

 Leaky ReLU: Allows small negative values to prevent "dying neuron"

issue.

 Softmax Function: Used for multi-class classification.

4. Importance in Neural Networks:

o Introduces non-linearity.

o Prevents vanishing gradient problems.

o Improves learning efficiency.

Q1b) Explain the architecture of an Artificial Neural Network with a neat diagram.

Answer:

1. Definition:

o An Artificial Neural Network (ANN) is a computational system inspired by the

human brain.
 o Consists of layers of neurons connected through weighted links.

2. Components of ANN:

o Input Layer: Receives raw input data.

o Hidden Layer(s): Processes data using activation functions.

o Output Layer: Produces final predictions or classifications.

3. Working:

o Inputs are multiplied by weights and passed through an activation function.

o Hidden layers transform data using non-linear functions.

o Output layer produces the final decision.

4. Diagram:

o Include layers (input, hidden, output) and interconnections.

5. Applications:

o Image recognition, speech processing, self-driving cars, etc.

Q1c) Explain Mc-Culloch & Pitts model with an example.

Answer:

1. Introduction:

o Developed in 1943 as the first artificial neuron model.

o Based on binary threshold logic.

2. Structure:

o Inputs (x1,x2,x3,...xnx_1, x_2, x_3, ... x_nx1,x2,x3,...xn)

o Weights (w1,w2,w3,...wnw_1, w_2, w_3, ... w_nw1,w2,w3,...wn)

o Summation Function: Computes weighted sum of inputs.

o Threshold Activation: Outputs 1 if sum exceeds the threshold, else 0.

3. Mathematical Model:

y=f(∑wixi)y = f(\sum w_i x_i)y=f(∑wixi)

o Output is 1 if the weighted sum exceeds a threshold.

4. Example - AND Gate:

 o Inputs: x1,x2x_1, x_2x1,x2

o Weights: w1=1,w2=1w_1 = 1, w_2 = 1w1=1,w2=1

o Threshold: 2

o Output = 1 only when both inputs are 1.

5. Limitations:

o Cannot handle complex problems.

o No learning mechanism.

Q2a) Differences among McCulloch-Pitts, Perceptron, and Adaline Models

Answer:

McCulloch-
Feature Perceptron Adaline
Pitts

Year Introduced 1943 1958 1960

Activation
Step Function Step Function Linear Activation
Function

Learning Rule No learning Hebbian Learning Least Mean Squares (LMS)

Weight
Fixed Adjusted Adjusted using gradient descent
Adjustment

Linear Linear classification with error

Application Logic functions
classification minimization

Q2b) Explain the structure and working of a Biological Neural Network.

Answer:

1. Definition:

o A network of neurons in the human brain that processes and transmits

information.

2. Structure:

o Neuron: Basic unit of the nervous system.

o Dendrites: Receive signals from other neurons.

 o Axon: Transmits signals to the next neuron.

o Synapse: Junction for signal transmission.

3. Working Mechanism:

o Neurons receive signals.

o If the signal exceeds a threshold, it fires an action potential.

o The signal is transmitted to the next neuron.

4. Comparison with Artificial Neural Networks (ANNs):

o Biological neurons learn dynamically.

o ANNs are computational approximations.

5. Applications in AI:

o Inspired deep learning and pattern recognition.

Q2c) Differences between Biological and Artificial Neural Networks

Answer:

Feature Biological Neural Network Artificial Neural Network

Structure Made of neurons Made of artificial nodes

Learning Unsupervised, self-adaptive Supervised learning

Processing Parallel processing Sequential or parallel

Energy Efficiency Low energy consumption High computation cost

Fault Tolerance Highly robust Can fail if critical components fail

Q3a) Explain Perceptron Learning Algorithm with an example.

Answer:

1. Definition:

o A single-layer neural network for binary classification.

2. Steps:

o Initialize weights and bias.

o Compute weighted sum and apply activation function.

 o Adjust weights based on error.

3. Mathematical Representation:

y=f(∑wixi+b)y = f(\sum w_i x_i + b)y=f(∑wixi+b)

o Uses step activation function.

4. Example - OR Gate:

o Inputs: (0,0), (0,1), (1,0), (1,1)

o Target: 0, 1, 1, 1

o Weights adjusted iteratively.

5. Limitations:

o Cannot solve non-linearly separable problems.

Q3b) Explain the architecture of a Multilayered Neural Network.

Answer:

1. Definition:

o A neural network with multiple hidden layers.

2. Components:

o Input Layer: Accepts raw data.

o Hidden Layers: Extracts features.

o Output Layer: Produces predictions.

3. Working:

o Inputs pass through multiple layers.

o Activation functions introduce non-linearity.

o Backpropagation adjusts weights.

4. Advantages:

o Solves complex problems.

o Improves accuracy.

5. Applications:

o Speech recognition, image classification, etc.

Q3c) Steps of Backpropagation Learning Algorithm

Answer:

1. Initialization:

o Assign small random weights.

2. Forward Propagation:

o Compute output layer activations.

3. Compute Error:

o Difference between actual and predicted output.

4. Backward Propagation:

o Calculate gradient of error.

o Adjust weights using gradient descent.

5. Repeat Until Convergence.

Q4a) Draw and Explain Backpropagation Network.

Answer:

1. Definition:

o A neural network using backpropagation for learning.

2. Structure:

o Input, hidden, and output layers.

o Uses differentiable activation functions.

3. Working:

o Forward propagation.

o Compute error.

o Backpropagate using gradients.

4. Advantages:

o Efficient weight optimization.

5. Applications:
 o Deep learning, pattern recognition.

Q4b) Differentiate between Feedforward and Feedback Neural Networks.

Answer:

Feature Feedforward Network Feedback Network

Direction of Flow One-way (input to output) Cyclic, feedback loops

Memory No memory Has memory

Example Multilayer Perceptron (MLP) Recurrent Neural Network (RNN)

Q4c) What is Error Correction and How to Minimize It?

Answer:

1. Definition:

o Process of reducing prediction errors in neural networks.

2. Techniques:

o Gradient Descent.

o Weight Regularization.

o Adaptive Learning Rate.

3. Importance:

o Improves accuracy.

4. Methods:

o Use dropout to prevent overfitting.

5. Conclusion:

o Error minimization is key for optimal performance.

UNIT 2
Q1a) What is the role of activation function in neural networks? Explain bipolar Sigmoid
function in detail.

Answer:

1. Role of Activation Function:

o Introduces non-linearity in neural networks.

o Helps neurons decide whether to activate or not.

o Allows deep networks to learn complex patterns.

o Prevents the vanishing gradient problem in deep networks.

2. Bipolar Sigmoid Function:

o A variation of the sigmoid activation function.

o Output ranges between -1 and +1 instead of 0 and 1.

o Formula: f(x)=1−e−x1+e−xf(x) = \frac{1 - e^{-x}}{1 + e^{-x}}f(x)=1+e−x1−e−x

o Derivative: f′(x)=12(1−f(x)2)f'(x) = \frac{1}{2} (1 - f(x)^2)f′(x)=21(1−f(x)2)

3. Advantages:

o Solves vanishing gradient problem better than standard sigmoid.

o Useful in training networks with bipolar inputs.

4. Disadvantages:

o Still suffers from saturation issues.

o Computation is expensive due to exponentiation.

5. Applications:

o Used in control systems and pattern recognition tasks.

Q1b) Why is ReLU the most commonly used Activation Function?

Answer:

1. Definition of ReLU (Rectified Linear Unit):

o Formula: f(x)=max(0,x)f(x) = max(0, x)f(x)=max(0,x)

o Output is 0 for negative inputs and x for positive inputs.

2. Reasons for Popularity:

o Efficient Computation: Simple max operation speeds up training.

o Avoids Vanishing Gradient: Unlike sigmoid and tanh, it does not saturate in
positive regions.

o Sparsity: Outputs zero for negative values, reducing unnecessary

computations.

o Better Convergence: Faster learning and improved accuracy in deep

networks.

3. Disadvantages:

o Dying ReLU Problem: Neurons can become inactive if inputs remain negative.

o Solution: Leaky ReLU and Parametric ReLU.

4. Where is ReLU Used?

o Deep learning architectures (CNNs, RNNs, GANs, etc.).

o Image recognition and natural language processing.

Q1c) Explain the architecture of an Artificial Neural Network with a neat diagram.

Answer:

1. Definition:

o A computational model inspired by the human brain.

o Consists of interconnected neurons arranged in layers.

2. Layers of ANN:

o Input Layer: Accepts raw input features.

o Hidden Layers: Processes inputs using weighted connections.

o Output Layer: Produces final predictions.

3. Working Mechanism:
 o Inputs are multiplied by weights and summed.

o Activation function determines neuron activation.

o Output layer generates the final prediction.

4. Diagram:

o Include an input layer, hidden layers, and an output layer with weighted
connections.

5. Applications:

o Image recognition, speech processing, medical diagnosis.

Q2a) Draw the structure of the biological neuron and explain its working briefly.

Answer:

1. Structure of Biological Neuron:

o Dendrites: Receive signals from other neurons.

o Cell Body (Soma): Processes incoming signals.

o Axon: Transmits signals to other neurons.

o Synapse: Junction where signals pass to the next neuron.

2. Working Mechanism:

o Dendrites collect electrical signals from other neurons.

o If the combined signal exceeds a threshold, the neuron fires.

o Signal travels along the axon to synapses, transmitting to the next neuron.

3. Significance:

o Basis for all human cognition and decision-making.

o Inspired artificial neural networks.

4. Comparison with ANN:

o Biological neurons are much more complex than artificial neurons.

5. Applications:

o Neuroscience, AI, brain-machine interfaces.

Q2b) Algorithm of ADALINE and its upper bound with largest Eigen Value of its correlation
matrix.

Answer:

1. ADALINE (Adaptive Linear Neuron) Algorithm:

o Similar to the perceptron but uses Mean Squared Error (MSE) for weight
updates.

o Learns a linear decision boundary.

2. Steps:

1. Initialize small random weights.

2. Compute the weighted sum: y=∑wixiy = \sum w_i x_iy=∑wixi

3. Apply activation function (linear).

4. Compute error: e=ytarget−ye = y_{\text{target}} - ye=ytarget−y

5. Update weights using: wi=wi+ηexiw_i = w_i + \eta e x_iwi=wi+ηexi

3. Eigenvalue Upper Bound:

o The largest eigenvalue of the correlation matrix determines learning speed.

o Upper bound ensures weight convergence.

4. Advantages:

o More stable than perceptron.

o Can handle linearly separable problems.

5. Limitations:

o Cannot classify non-linearly separable problems.

Q3a) What is Error Correction and how to minimize these errors?

Answer:

1. Definition:

o Adjusting weights in neural networks to reduce prediction errors.

2. Types of Errors:

o Training Error: Difference between predicted and actual values.

o Generalization Error: Performance drop on unseen data.

 3. Error Minimization Techniques:

o Gradient Descent: Optimizes weights iteratively.

o Regularization (L1, L2): Prevents overfitting.

o Dropout: Reduces overfitting by randomly disabling neurons.

o Batch Normalization: Normalizes activations to stabilize learning.

4. Importance:

o Reduces loss function.

o Improves accuracy and stability.

5. Applications:

o Speech recognition, image classification.

Q3b) Explain the architecture of Multilayered Neural Network.

Answer:

1. Definition:

o A neural network with multiple hidden layers.

2. Layers:

o Input Layer: Takes raw data.

o Hidden Layers: Extract features.

o Output Layer: Makes predictions.

3. Working:

o Inputs propagate through hidden layers.

o Non-linearity is introduced via activation functions.

o Outputs are refined at each layer.

4. Advantages:

o Can model complex patterns.

o Handles non-linear problems.

5. Applications:

o Deep learning, speech processing.

Q3c) Define Learning and Memory. Explain learning algorithms in detail.

Answer:

1. Learning:

o Process of adjusting network weights based on data.

2. Memory:

o Retaining learned patterns for future predictions.

3. Types of Learning Algorithms:

o Supervised Learning: Uses labeled data.

o Unsupervised Learning: Finds hidden patterns in unlabeled data.

o Reinforcement Learning: Learns via rewards.

4. Examples:

o Hebbian Learning: "Neurons that fire together, wire together."

o Gradient Descent: Optimizes weights via error minimization.

5. Applications:

o AI assistants, self-driving cars.

Q4a) Difference between Forward Propagation and Backward Propagation.

Answer:

Feature Forward Propagation Backward Propagation

Definition Computes output from input Updates weights using gradients

Direction Input → Output Output → Input

Purpose Passes data through the network Minimizes error using weight updates

Computation Uses activation functions Uses gradient descent

Example Making predictions Adjusting weights in training

Q4b) Explain different types of Gradient Descent in detail.

Answer:
 1. Batch Gradient Descent (BGD): Uses entire dataset.

2. Stochastic Gradient Descent (SGD): Uses one sample at a time.

3. Mini-Batch Gradient Descent: Uses small batches.

4. Momentum-Based GD: Uses past gradients to accelerate learning.

5. Adam Optimizer: Combines momentum and adaptive learning rates.

Q4c) Perceptron Learning Algorithm for OR function with calculations.

1. Inputs & Outputs:

o OR Truth Table.

2. Steps:

o Initialize weights.

o Compute output.

o Adjust weights iteratively.

3. Calculations:

o Perform updates for each epoch.

4. Result:

o Final weight values classify OR function.