.Section 6.

2 Review Questions and Answers:

1. What is deep learning? What can deep learning do?

o Deep Learning: Deep learning is a subset of machine learning within artificial
intelligence (AI) that involves algorithms inspired by the structure and function of
the brain's neural networks. It utilizes multiple layers of interconnected neurons to
process and learn from large amounts of data, automatically extracting complex
features and patterns.
o Capabilities: Deep learning can perform various tasks, including:
 Image and Video Recognition: Identifying objects, faces, and actions in
images and videos.
 Speech and Audio Processing: Recognizing and generating human
speech.
 Natural Language Processing (NLP): Understanding, translating, and
generating human language.
 Autonomous Systems: Powering self-driving cars and robots.
 Recommendation Systems: Providing personalized content suggestions.
 Medical Diagnosis: Analyzing medical images and data for disease
detection and prognosis.
2. Compared to traditional machine learning, what is the most prominent difference of
deep learning?
o 45mProminent Difference: The most prominent difference is deep learning's
ability to automatically extract and learn complex features from raw data without
the need for manual feature engineering. Traditional machine learning algorithms
require human experts to design and input relevant features, whereas deep
learning algorithms can discover and represent these features through multiple
layers of abstraction.
3. List and briefly explain different learning methods in AI.
o Supervised Learning: Involves training a model on ,
where the input data is paired with the correct output. The model learns to map
inputs to outputs and is used for tasks such as classification and regression.
o Unsupervised Learning: Involves training a model on unlabeled data, where the
model tries to find patterns and structures in the data without predefined labels.
Common tasks include clustering and dimensionality reduction.
o Semi-Supervised Learning: Combines a small amount of labeled data with a
large amount of unlabeled data during training. It aims to improve learning
accuracy by leveraging both types of data.
o Reinforcement Learning: Involves training an agent to make a sequence of
decisions by rewarding it for desirable actions and punishing it for undesirable
ones. This method is used in applications such as game playing and robotics.
4. What is representation learning, and how does it relate to deep learning?
o Representation Learning: Representation learning is a type of machine learning
where the algorithm learns to automatically discover the representations needed
for feature detection or classification from raw data. It enables the model to learn
the optimal features needed for the task directly from the data.
 o Relation to Deep Learning: Deep learning is a subset of representation learning.
Deep learning models, such as deep neural networks, automatically learn
hierarchical representations of the data through multiple layers. Each layer
captures different levels of abstraction, from simple edges and textures in images
to more complex shapes and patterns. This capability is what allows deep learning
models to excel at tasks like image and speech recognition, where traditional
machine learning approaches struggle to manually define relevant features.

Multiple Choice Questions (MCQs) with Answers:

1. What is deep learning?

o A) A subset of unsupervised learning
o B) A technique to manually create features
o C) A subset of machine learning that uses neural networks with multiple layers
o D) A method of statistical analysis
o Answer: C
2. Which of the following tasks is deep learning particularly good at?
o A) Basic arithmetic operations
o B) Image and speech recognition
o C) Linear regression
o D) Sorting algorithms
o Answer: B
3. What is the most prominent difference between traditional machine learning and
deep learning?
o A) Deep learning requires more manual feature engineering.
o B) Traditional machine learning algorithms are better at handling unstructured
data.
o C) Deep learning can automatically extract and learn complex features from raw
data.
o D) Traditional machine learning algorithms are more computationally intensive.
o Answer: C
4. Which learning method involves training a model on labeled data?
o A) Unsupervised Learning
o B) Reinforcement Learning
o C) Semi-Supervised Learning
o D) Supervised Learning
o Answer: D
5. What does reinforcement learning involve?
o A) Training on labeled data
o B) Training on unlabeled data
o C) Training an agent to make decisions by rewarding desirable actions
o D) Training on a combination of labeled and unlabeled data
o Answer: C
6. Representation learning focuses on:
o A) Manually creating features for the model
 o B) Automatically discovering the representations needed for feature detection or
classification
o C) Training on large amounts of labeled data
o D) Using statistical methods to analyze data
o Answer: B
7. In which scenario is deep learning preferred over traditional machine learning?
o A) When dealing with small, structured datasets
o B) When manual feature engineering is straightforward
o C) When dealing with large, unstructured datasets like images and speech
o D) When simple linear models are sufficient
o Answer: C
8. Which of the following is not a common application of deep learning?
o A) Image recognition
o B) Speech recognition
o C) Simple arithmetic operations
o D) Natural language processing
o Answer: C
9. Which AI learning method typically involves clustering and dimensionality
reduction tasks?
o A) Supervised Learning
o B) Unsupervised Learning
o C) Semi-Supervised Learning
o D) Reinforcement Learning
o Answer: B
10. What is a common challenge in supervised learning for deep networks?
o A) Lack of algorithms
o B) Computational simplicity
o C) Labeling large datasets
o D) Low accuracy
o Answer: C
11. Which of the following best describes a "knowledge-based system"?
o A) A system that learns features automatically
o B) A system that relies on human-created features
o C) A system that uses deep learning techniques
o D) A system that performs unsupervised learning
o Answer: B
12. The initial idea of deep learning dates back to:
o A) Early 2000s
o B) Late 1980s
o C) Mid-1990s
o D) Early 2010s
o Answer: B
13. What did the research of LeCun and colleagues in 1989 focus on?
o A) Developing reinforcement learning algorithms
o B) Applying backpropagation networks for handwritten ZIP code recognition
 o C) Creating unsupervised learning models
o D) Implementing simple decision trees
o Answer: B
14. Deep learning models require:
o A) Small datasets and low computational power
o B) Large datasets and high computational power
o C) No data preprocessing
o D) Only numerical data
o Answer: B
15. Which of the following best describes "semi-supervised learning"?
o A) Training with labeled data only
o B) Training with unlabeled data only
o C) Combining a small amount of labeled data with a large amount of unlabeled
data
o D) Reinforcement through rewards and punishments
o Answer: C

Section 6.3 Review Questions and Answers:

1. How does a single artificial neuron (i.e., PE) work?

o Functioning of a Single Neuron:
 Input (p): Receives a numerical input.
 Weight (w): Each input is multiplied by an adjustable weight.
 Bias (b): A bias term is added to the weighted input.
 Net Input Function (g): Computes the net input n=wp+bn = wp +
bn=wp+b.
 Transfer (Activation) Function (f): Transforms the net input nnn to
produce the output a=f(n)a = f(n)a=f(n).
 Example Calculation: If w=2w = 2w=2, p=3p = 3p=3, and b=−1b = -
1b=−1, then a=f(2×3−1)=f(5)a = f(2 \times 3 - 1) = f(5)a=f(2×3−1)=f(5).
2. List and briefly describe the most commonly used ANN activation functions.
o Linear: The output is a linear function of the input. Simple but not suitable for
capturing non-linear patterns.
o Sigmoid: f(x)=11+e−xf(x) = \frac{1}{1 + e^{-x}}f(x)=1+e−x1. Outputs a value
between 0 and 1, useful for binary classification.
o Hyperbolic Tangent (tanh): f(x)=ex−e−xex+e−xf(x) = \frac{e^x - e^{-x}}{e^x
+ e^{-x}}f(x)=ex+e−xex−e−x. Outputs values between -1 and 1, often used for
hidden layers.
o ReLU (Rectified Linear Unit): f(x)=max⁡(0,x)f(x) = \max(0, x)f(x)=max(0,x).
Introduces non-linearity while being computationally efficient, widely used in
deep learning.
o Leaky ReLU: A variant of ReLU that allows a small gradient when the unit is not
active, preventing dying neurons.
o Softmax: Converts logits into probabilities, used in the output layer for multi-
class classification problems.
3. What is MLP, and how does it work?
 o Multilayer Perceptron (MLP):
 Architecture: Consists of multiple layers of neurons: an input layer, one
or more hidden layers, and an output layer.
 Feedforward: Information flows forward from input to output without
feedback loops.
 Training: Adjusts weights and biases using a training algorithm like
backpropagation to minimize the error between predicted and actual
outputs.
 Activation Functions: Each neuron applies an activation function to its
input to produce an output.
4. Explain the function of weights in ANN.
o Function of Weights:
 Connection Strength: Weights determine the strength and importance of
inputs to a neuron.
 Learning: During training, weights are adjusted to minimize the error in
predictions, storing the learned patterns.
 Influence: The product of an input and its weight influences the neuron's
net input, affecting the neuron's output.
5. Describe the summation and activation functions in MLP-type ANN architecture.
o Summation Function:
 Definition: Computes the weighted sum of inputs for a neuron.
 Formula: n=∑(wi×pi)+bn = \sum (w_i \times p_i) + bn=∑(wi×pi)+b,
where pip_ipi are inputs, wiw_iwi are weights, and bbb is the bias.
 Role: Aggregates the inputs to determine the neuron's level of activation.
o Activation Function:
 Definition: Transforms the net input to produce the neuron's output.
 Types: Common activation functions include sigmoid, tanh, ReLU, and
softmax.
 Purpose: Introduces non-linearity, enabling the network to learn and
model complex patterns and relationships in the data.

Section 6.4 Review Questions and Answers:

1. List the nine steps in conducting a neural network project.

o Define the Project Objectives: Clearly outline what you aim to achieve with the
neural network project.
o Collect and Prepare the Data: Gather data relevant to the problem and
preprocess it to make it suitable for training.
o Partition the Data: Split the data into training, validation, and test sets.
o Select the Neural Network Architecture: Choose the appropriate type and
structure of the neural network.
o Set the Network Parameters and Initialize Weights: Define parameters such as
learning rate, momentum, number of layers, and initialize weights.
o Train the Network: Use the training data to adjust weights and biases to
minimize the error.
 o Validate the Network: Evaluate the network's performance using the validation
data to fine-tune parameters and avoid overfitting.
o Test the Network: Assess the final performance of the network using the test
data to estimate its generalization capability.
o Deploy and Monitor the Network: Implement the trained network in a real-
world environment and monitor its performance over time.
2. What are some of the design parameters for developing a neural network?
o Learning Rate: Determines how much the weights are adjusted during training.
o Momentum: Helps accelerate gradients vectors in the right directions, thus
leading to faster converging.
o Number of Layers: Decides the depth of the network; can include input, hidden,
and output layers.
o Number of Neurons per Layer: Defines the width of each layer in the network.
o Activation Functions: The functions used to introduce non-linearity into the
model (e.g., sigmoid, ReLU).
o Training Algorithm: The method used for optimizing the weights (e.g., gradient
descent, Adam).
o Initialization of Weights: The initial values for the network weights.
o Regularization Techniques: Methods like dropout or L2 regularization to
prevent overfitting.
o Batch Size: Number of training samples used in one iteration before updating the
model's parameters.
3. Draw and briefly explain the three-step process of learning in ANN.
o Feedforward Pass: Inputs are passed through the network layer by layer, and
outputs are calculated.
o Backpropagation: The error (difference between predicted and actual outputs) is
propagated back through the network to update the weights.
o Weight Update: Adjust weights using optimization algorithms (e.g., gradient
descent) to minimize the error.

plaintext