DEEP LEARNING

ASSESSMENT – I
UNIT – I
Q1: Discuss the evolution of Artificial Intelligence (AI) and its connection to early neural networks.
How did early neural networks lay the foundation for modern deep learning?

Q2: What is probabilistic modeling in machine learning? Explain its significance and how it differs
from deterministic approaches.

Q3: Compare kernel methods and decision trees in machine learning. What are the key strengths and
limitations of each approach?

Q4: Define the four branches of machine learning (Supervised, Unsupervised, Semi-supervised, and
Reinforcement Learning) and provide an example use case for each.

Q5: What are overfitting and underfitting in machine learning? Discuss strategies to prevent each
issue during model training.

Q6: Why is evaluating machine learning models important? Discuss precision, recall, and F1-score as
evaluation metrics and their relevance in real-world applications.

UNIT – II
Q1: How does biological vision inspire the design of machine vision systems? Explain the similarities
and differences between the two.

Q2: Compare human language processing with machine language processing. How has deep learning
enhanced natural language understanding?

Q3: What is an artificial neural network (ANN)? Describe its basic structure and explain how it
mimics the functioning of the human brain.

Q4: What are the key steps involved in training a deep neural network? Discuss the role of loss
functions, optimization, and backpropagation in this process.

Q5: What is the backpropagation algorithm? How is it used for training a neural network?

Q6: What are the primary challenges in training and improving deep networks? How can issues like
vanishing gradients and overfitting be addressed?

UNIT – III
Q1: What are the primary components of a neural network, and how do they contribute to its
functioning? Discuss layers, weights, activation functions, and biases.
Q2: What is Keras, and how does it simplify the process of building neural networks? Compare Keras
with TensorFlow, Theano, and CNTK.

Q3: What are the key hardware and software requirements for setting up a deep learning workstation?
Explain the role of GPUs in accelerating deep learning tasks.

Q4: How can a neural network be designed for binary classification tasks such as classifying movie
reviews as positive or negative? Discuss the choice of loss functions and activation functions.

Q5: What are the challenges in multiclass classification tasks like classifying newswires? Explain
how to design a neural network for such tasks, including the use of softmax activation.

Q6: Compare TensorFlow, Theano, and CNTK in terms of their features, performance, and
use cases.
Q. Compare TensorFlow, Theano, and CNTK in terms of their features,
performance, and use cases.

Comparison of TensorFlow, Theano, and CNTK

CNTK (Microsoft
Aspect TensorFlow Theano
Cognitive Toolkit)
Developed by
Developed and Université de Montréal Developed and maintained
Development
maintained by Google. (no longer actively by Microsoft.
maintained).
Very popular and widely Used mainly in
Gained some traction but
used for both research research, now largely
Popularity not as widely adopted as
and production obsolete due to lack of
TensorFlow.
purposes. maintenance.
High performance with Efficient, but less
Optimized for large-scale
strong GPU/TPU optimized for large-
Performance distributed processing with
support and distributed scale production
good GPU support.
computing. environments.
Low-level API,
Offers both low-level requiring detailed API is relatively simple but
Ease of Use control and high-level knowledge of not as user-friendly as
abstractions (via Keras). mathematics and TensorFlow/Keras.
programming.
Extensive ecosystem
Focused on symbolic Excellent for speech
(TensorBoard,
Features differentiation and low- recognition and other
TensorFlow Lite,
level optimization. specific use cases.
TensorFlow.js, etc.).
Highly scalable with
Not designed for large-
distributed training and Scalable, particularly in
Scalability scale distributed
production-level cloud environments.
systems.
deployment.
Extremely flexible;
Limited flexibility Focused on specific
supports multiple
Flexibility compared to applications like speech
backends and advanced
TensorFlow. and text processing.
customizations.
Actively developed with Active but overshadowed
Current Deprecated; no longer
regular updates and a by TensorFlow and
Status maintained since 2017.
large community. PyTorch.
Q. What is the backpropagation algorithm? How is it used for training a
neural network?

Backpropagation Algorithm: An Overview

Backpropagation (short for backward propagation of errors) is a supervised learning

algorithm used for training artificial neural networks. It is an efficient method for calculating
gradients and updating model parameters (weights and biases) to minimize the error between
predicted and actual outputs.

Steps in the Backpropagation Algorithm

The backpropagation process consists of the following key steps:

1. Forward Propagation

 The input data is passed through the neural network, layer by layer.
 At each layer, weights and biases are applied to calculate the outputs (activations) of the
neurons.
 The output of the final layer is compared to the target values, resulting in a loss/error.

2. Compute the Loss

 A loss function (e.g., Mean Squared Error, Cross-Entropy) is used to quantify the difference
between the predicted output and the actual target.

3. Backward Propagation

 Error Calculation:
The error signal is propagated backward from the output layer to the input layer.
 Gradient Calculation:

Using the chain rule of calculus, the gradient (partial derivatives) of the loss function with
respect to each weight and bias is calculated. This involves two main parts:

o Output Layer: Calculate the gradient of the loss with respect to the output layer
weights and biases.
o Hidden Layers: Calculate the gradient for each hidden layer recursively by
propagating the error backward.
5. Iteration

 The process of forward propagation, loss computation, backpropagation, and weight updating
is repeated for several iterations (epochs) until the network converges (i.e., the loss becomes
sufficiently small).

How Backpropagation is Used for Training Neural Networks

1. Initialization:
o Initialize the weights and biases with small random values.
2. Training Process:
o Feed the input data through the network (forward propagation).
o Calculate the loss using a loss function.
o Propagate the error backward through the network (backpropagation).
o Update the weights and biases using the gradients.
3. Stopping Criteria:
o Training is stopped when:
 The loss becomes very small (convergence).
 The model reaches a specified number of epochs.
 The validation performance stops improving (to avoid overfitting).
Q. What are the key hardware and software requirements for setting up a deep learning
workstation? Explain the role of GPUs in accelerating deep learning tasks.

Key Hardware and Software Requirements for Setting up a Deep Learning

Workstation

Setting up a deep learning workstation requires a balance of hardware components and

software tools to handle the computational demands of training deep neural networks
efficiently.

Hardware Requirements

1. Central Processing Unit (CPU)

o Requirement: A high-performance multi-core processor.
o Reason: Handles preprocessing tasks, loading data, managing I/O, and
controlling GPUs.
o Examples: Intel Core i7/i9, AMD Ryzen 9, AMD Threadripper.
2. Graphics Processing Unit (GPU)
o Requirement: At least one powerful GPU with high CUDA core counts and
large memory.
o Reason: Accelerates deep learning tasks such as matrix multiplications, tensor
operations, and gradient computations.
o Examples: NVIDIA RTX 3090, A100, or NVIDIA Quadro series
(recommended for deep learning due to CUDA support).
3. Random Access Memory (RAM)
o Requirement: At least 16 GB, preferably 32 GB or more.
o Reason: Ensures smooth data preprocessing and loading, particularly for large
datasets.
4. Storage
o Requirement:
 SSD (Solid State Drive): For fast read/write speeds (minimum 500 GB,
ideally 1 TB or more).
 HDD: For storing large datasets and model checkpoints (2 TB or
more).
o Reason: SSDs speed up data access, while HDDs offer economical storage for
massive data.
5. Motherboard
o Requirement: A motherboard compatible with the chosen CPU and GPU(s).
o Reason: Must support PCIe slots for GPUs and sufficient power delivery.
6. Power Supply Unit (PSU)
o Requirement: High-wattage PSU (750W or more) depending on the number
of GPUs and system requirements.
o Reason: Ensures stable power delivery for all components, especially GPUs.
7. Cooling System
o Requirement: Adequate air or liquid cooling for CPU and GPU.
o Reason: Prevents thermal throttling during prolonged training.
 8. Display and Peripherals
o Requirement: High-resolution monitor and a comfortable keyboard/mouse
setup.
o Reason: For easier model visualization and workstation interaction.

Software Requirements

1. Operating System (OS)

o Requirement: Ubuntu (Linux) is preferred, though Windows or macOS are
also compatible.
o Reason: Linux is optimized for performance and supported by most deep
learning libraries.
2. Programming Language
o Requirement: Python (preferred), R, or C++.
o Reason: Python offers extensive libraries and frameworks for deep learning.
3. Deep Learning Frameworks
o Requirement:
 TensorFlow (supports both CPUs and GPUs).
 PyTorch (widely used and GPU-friendly).
 Keras (high-level API).
o Reason: Provide pre-built functions and tools for building and training neural
networks.
4. GPU Drivers and Libraries
o Requirement: NVIDIA drivers, CUDA toolkit, and cuDNN library.
o Reason: Ensures optimal GPU acceleration for deep learning computations.
5. Integrated Development Environments (IDEs)
o Requirement: Jupyter Notebook, PyCharm, VS Code.
o Reason: Simplify coding, debugging, and visualization.
6. Data Libraries
o Requirement: NumPy, Pandas, Matplotlib, OpenCV, etc.
o Reason: Enable data manipulation and visualization.
7. Version Control Systems
o Requirement: Git.
o Reason: For code versioning and collaborative development.

Role of GPUs in Accelerating Deep Learning Tasks

1. Parallel Processing Capability

o GPUs consist of thousands of cores designed for parallelism, making them
ideal for performing multiple operations simultaneously.
o Example: GPUs handle tensor operations, which are computationally
expensive, like matrix multiplications and convolutions in neural networks.
2. Faster Training Times
o Neural networks involve iterative computations over large datasets, which can
take days on CPUs. GPUs significantly reduce this time by parallelizing the
workload.
 o Example: Training ResNet-50 on ImageNet might take weeks on a CPU but
only hours on a GPU.
3. Memory Bandwidth
o GPUs provide high memory bandwidth for transferring data between the
processor and memory quickly.
o Example: High-end GPUs like NVIDIA A100 have HBM2 memory for faster
data access.
4. Efficient Floating-Point Operations
o GPUs are optimized for performing large-scale floating-point calculations.
o Example: Training large models such as GPT or BERT requires FP32 or
mixed-precision FP16 calculations, which GPUs excel at.
5. Specialized Hardware for AI
o Modern GPUs like NVIDIA’s Ampere series include Tensor Cores,
specifically designed to accelerate AI and deep learning workloads.
6. Cost-Effectiveness for Large-Scale Workloads
o GPUs provide significantly better performance-to-cost ratios for deep learning
compared to CPUs.
o Example: A single NVIDIA RTX 3090 can outperform a cluster of CPUs for
deep learning tasks.