Introduction to ANN

1. What is an Artificial Neural Network (ANN)? Explain its basic

structure and components.
o Artificial Neural Network (ANN): A computational model inspired
by the structure and functioning of biological neural networks. ANNs
are used for tasks like pattern recognition, classification, regression,
and more.
o Basic Structure:

 Input Layer: Takes input features from the data.

 Hidden Layers: Intermediate layers where computations are
performed. The number of layers depends on the complexity
of the task.
 Output Layer: Produces the final output (e.g., predictions or
classifications).
o Components:

 Neurons: Basic units of computation, mimicking biological

neurons.
 Weights: Influence the strength of connections between
neurons.
 Biases: Allow the model to shift the activation function.
 Activation Functions: Introduce non-linearity into the
network.
2. Compare biological neurons with artificial neurons.
o Biological Neurons:

 Have dendrites to receive signals, a cell body to process

them, and an axon to transmit the signal to other neurons.
 Operate through electrochemical signals.
o Artificial Neurons:

 Receive inputs as numbers, apply weights and biases,

process them using a mathematical function, and pass the
result to the next layer.
 Operate through mathematical computations.
3. What are the roles of weights and biases in an ANN?
o Weights: Represent the strength of connections between neurons.
Larger weights indicate a stronger influence of the input feature or
neuron on the output.
o Biases: Allow the activation function to shift its output, enabling
the model to fit data that does not pass through the origin.
 4. Explain the concept of forward propagation in an ANN.
o Forward propagation is the process of passing input data through
the layers of the network to calculate the output.
 Steps:
1. Inputs are multiplied by weights and added to biases.
2. The result is passed through an activation function.
3. The output of one layer becomes the input to the next
layer.
4. This process continues until the output layer is
reached.
5. What is the difference between shallow and deep neural
networks?
o Shallow Neural Networks: Contain one or two hidden layers.
Suitable for simpler problems.
o Deep Neural Networks (DNN): Contain multiple hidden layers,
enabling the network to learn hierarchical features and solve
complex problems like image recognition or NLP tasks.

Learning in ANN
6. Describe the steps involved in training an ANN using the
backpropagation algorithm.
Backpropagation is a supervised learning algorithm used to minimize the
error in an ANN by adjusting weights and biases.
o Steps:

1. Initialization: Assign random values to weights and biases.

2. Forward Propagation: Pass input data through the network
to compute the output.
3. Compute Loss: Compare the predicted output with the
actual target using a loss function.
4. Backward Propagation: Calculate the gradient of the loss
function with respect to weights and biases using the chain
rule.
 Adjust weights and biases to minimize the loss.
5. Update Weights: Use an optimization algorithm like
Gradient Descent to update weights and biases iteratively.
6. Repeat: Continue steps 2–5 for multiple iterations (epochs)
until the loss converges to an acceptable value.
7. Explain the concept of gradient descent and its role in optimizing
neural networks.
o Gradient Descent: A mathematical optimization algorithm used to
minimize the loss function by iteratively updating model parameters
(weights and biases) in the direction of the steepest descent
(negative gradient).
 Steps:
1. Compute the gradient of the loss function with respect
to weights and biases.
2. Update parameters:
w:=w−η⋅∇L(w)w := w - \eta \cdot \nabla L(w)

∇L(w)\nabla L(w) is the gradient.

where ww is the weight, η\eta is the learning rate, and

o Role in Neural Networks:

 Ensures weights are adjusted to reduce the error.

 Helps the network learn patterns in the data efficiently.
8. What are the challenges of overfitting in neural networks, and
how can it be mitigated?
o Overfitting: Occurs when the model learns not only the patterns
but also the noise in the training data, leading to poor
generalization on unseen data.
o Mitigation Strategies:

 Regularization: Add penalties to the loss function (e.g., L1

or L2 regularization).
 Dropout: Randomly deactivate neurons during training to
prevent reliance on specific neurons.
 Early Stopping: Stop training when the validation error
stops improving.
 Data Augmentation: Increase the size and variability of the
training dataset.
 Reduce Model Complexity: Use fewer layers or neurons in
the network.
9. Differentiate between supervised, unsupervised, and
reinforcement learning in the context of ANNs.
o Supervised Learning:

 Data includes input-output pairs.

 ANN learns a mapping function from inputs to outputs.
 Example: Image classification, where each image has a label.
 o Unsupervised Learning:

 Data includes only inputs, and the ANN learns patterns or

structures in the data.
 Example: Clustering or dimensionality reduction using
Autoencoders.
o Reinforcement Learning:

 ANN learns by interacting with an environment and receiving

feedback in the form of rewards or penalties.
 Example: Training an agent to play a game or control a robot.

Activation Functions
10.Why are activation functions used in neural networks?
 Activation functions introduce non-linearity into the network, enabling it to
learn complex patterns and relationships. Without them, the model would
behave like a linear function, limiting its capacity to model intricate data.
 They also help normalize the output of neurons, keeping values
manageable during computations.
11.Explain the following activation functions and their significance:
 Sigmoid:
f(x)=11+e−xf(x) = \frac{1}{1 + e^{-x}}
o Outputs values in the range (0, 1).

o Used in binary classification problems.

o Significance: Smooth and differentiable but suffers from the

vanishing gradient problem.
 Tanh (Hyperbolic Tangent):
f(x)=ex−e−xex+e−xf(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}
o Outputs values in the range (-1, 1).

o Significance: Zero-centered, which often leads to faster

convergence compared to Sigmoid.
 ReLU (Rectified Linear Unit):
f(x)=max⁡(0,x)f(x) = \max(0, x)
o Outputs zero for negative inputs and linear for positive inputs.

o Significance: Efficient and mitigates the vanishing gradient

problem. However, it can suffer from the "dying ReLU" problem.
 Leaky ReLU:
f(x)=x if x>0 else 0.01xf(x) = x \text{ if } x > 0 \text{ else } 0.01x
 o Introduces a small slope for negative inputs.

o Significance: Addresses the dying ReLU issue by allowing a small

gradient for negative inputs.
 Softmax:
f(xi)=exi∑j=1nexjf(x_i) = \frac{e^{x_i}}{\sum_{j=1}^n e^{x_j}}
o Converts logits into probabilities that sum to 1.

o Significance: Commonly used in multi-class classification for

output layers.
12.Compare linear and non-linear activation functions. Why is non-
linearity important?
 Linear Activation Functions: Output is a linear transformation of the
input.
o Limitation: Cannot model complex relationships or separate non-
linear data.
 Non-linear Activation Functions: Introduce non-linearity, allowing the
network to learn complex patterns.
o Importance: Enable the stacking of layers to create deep networks
that can solve intricate problems.
13.Discuss the vanishing gradient problem and how activation
functions like ReLU address it.
 Vanishing Gradient Problem: In deep networks, gradients become
extremely small in earlier layers during backpropagation, slowing learning.
This often occurs with activation functions like Sigmoid and Tanh.
 ReLU's Solution: ReLU does not saturate for positive values, maintaining
a gradient of 1 for most of its range, thus avoiding the vanishing gradient
issue.
14.Explain the difference between softmax and sigmoid in multi-class
classification problems.
 Softmax: Used when classes are mutually exclusive. Outputs a probability
distribution across all classes.
 Sigmoid: Used when classes are not mutually exclusive. Outputs
independent probabilities for each class.
15.What is the significance of choosing an appropriate activation
function for hidden layers vs. output layers?
 Hidden Layers: Functions like ReLU or Tanh are chosen for their ability to
introduce non-linearity and promote learning.
 Output Layers:
o Sigmoid for binary classification.
 o Softmax for multi-class classification.

o Linear activation for regression tasks.

Applications
16.List real-world applications of ANNs in various fields.
 Healthcare: Disease diagnosis, medical imaging, and drug discovery.
 Finance: Fraud detection, stock price prediction, and credit scoring.
 Autonomous Vehicles: Object detection, path planning, and sensor
fusion.
 Natural Language Processing: Sentiment analysis, machine
translation, and chatbots.
 Gaming: AI opponents and environment interaction.
17.What are the advantages and limitations of ANNs compared to
traditional machine learning algorithms?
 Advantages:
o Can model complex, non-linear relationships.

o Handles large datasets effectively.

o Learns directly from raw data, reducing the need for manual feature
engineering.
 Limitations:
o Computationally intensive and requires significant resources.

o Prone to overfitting without sufficient data or regularization.

o Lack of interpretability compared to simpler models like decision

trees.