Scribd
Upload
3 views4 pages

Aiml-Qb - Unit 5

nigger

Uploaded by

windsurfmahee1
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
3 views4 pages

Aiml-Qb - Unit 5

nigger

Uploaded by

windsurfmahee1
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 4

UNIT V NEURAL NETWORKS

Perceptron - Multilayer perceptron, activation functions, network training –


gradient descent optimization – stochastic gradient descent, error backpropagation,
from shallow networks to deep networks –Unit saturation (aka the vanishing gradient
problem) – ReLU, hyperparameter tuning, batch normalization, regularization,
dropout.

1. What is perceptron and its types?


A Perceptron is an Artificial Neuron. It is the simplest possible Neural Network. Neural
Networks are the building blocks of Machine Learning.

2. Draw the architecrute of multilayer perceptron?(APRIL/MAY 2023)

3. Name any two activation functions (APRIL/MAY 2023)


In MLP and CNN neural network models, ReLU is the default activation function for
hidden layers. In RNN neural network models, we use the sigmoid or tanh function for
hidden layers. The tanh function has better performance. Only the identity activation function
is considered linear.

4. Does MLP have activation function?


Multilayer perceptrons (MLP) has been proven to be very successful in many applications
including classification. The activation function is the source of the MLP power. Careful
selection of the activation function has a huge impact on the network performance.

5. Show the perceptron that calculates parity of its three inputs (Nov/Dec 2023)

6. What are the types of activation function?


Popular types of activation functions and when to use them
 Binary Step Function
 Linear Function
 Sigmoid
 Tanh
 ReLU
 Leaky ReLU
1
 Parameterised ReLU
 Exponential Linear Unit

7. What is MLP and how does it work?


A multilayer perceptron (MLP) is a feedforward artificial neural network that generates a
set of outputs from a set of inputs. An MLP is characterized by several layers of input nodes
connected as a directed graph between the input and output layers. MLP uses
backpropogation for training the network.
8. Differentiate computer and human brain ( Nov/Dev 2023)
Processing:
 Computer: Follows precise, logical instructions and operates on binary data (0s and 1s).
 Human Brain: Works through complex, parallel processing with neurons that fire based
on electrical and chemical signals, handling a wide range of tasks simultaneously.
Learning:
 Computer: Learns through explicit programming or training with large datasets (e.g.,
machine learning models).
 Human Brain: Learns through experience, intuition, and sensory input, capable of
adapting and generalizing from minimal examples with flexibility.

9. What are the advantages of Multilayer Perceptron?


Advantages of Multi-Layer Perceptron:
A multi-layered perceptron model can be used to solve complex non-linear problems.
It works well with both small and large input data.
It helps us to obtain quick predictions after the training.
It helps to obtain the same accuracy ratio with large as well as small data.

10. What do you mean by activation function?


An activation function is a function used in artificial neural networks which outputs a
small value for small inputs, and a larger value if its inputs exceed a threshold. If the inputs
are large enough, the activation function "fires", otherwise it does nothing.

11. How many layers are there in perceptron?


This is known as a two-layer perceptron. It consists of two layers of neurons. The first
layer is known as hidden layer, and the second layer, known as the output layer, consists of a
single neuron.
12. is stochastic gradient descent same as gradient descent?
Compared to Gradient Descent, Stochastic Gradient Descent is much faster, and more
suitable to large-scale datasets. But since the gradient it's not computed for the entire dataset,
and only for one random point on each iteration, the updates have a higher variance.

13. Does stochastic gradient descent lead to faster training?


Gradient Descent is the most common optimization algorithm and the foundation of how
we train an ML model. But it can be really slow for large datasets. That's why we use a
variant of this algorithm known as Stochastic Gradient Descent to make our model learn a lot
faster.

14. What is stochastic gradient descent and why is it used in the training of neural
networks?
Stochastic Gradient Descent is an optimization algorithm that can be used to train neural
network models. The Stochastic Gradient Descent algorithm requires gradients to be
calculated for each variable in the model so that new values for the variables can be
calculated.
2
15. What is stochastic gradient descent and why it is used in the training of neural
networks ? (April/May 2024)
Stochastic Gradient Descent (SGD) is an optimization algorithm used to minimize the loss
function during the training of neural networks. It is a variant of gradient descent where,
instead of using the entire dataset to compute the gradient, it uses a single randomly selected
training example (or a small batch) at each step.
Why it is used:
Efficiency: SGD is computationally more efficient because it updates the model parameters
after each training example or small batch, rather than waiting for the entire dataset.
Faster Convergence: Due to its stochastic nature, SGD can escape local minima and find
better solutions, often converging faster than batch gradient descent, especially with large
datasets.
16. What are the disadvantages of stochastic gradient descent?
SGD is much faster but the convergence path of SGD is noisier than that of original
gradient descent. This is because in each step it is not calculating the actual gradient but an
approximation. So we see a lot of fluctuations in the cost.

17. How do you solve the vanishing gradient problem within a deep neural network?
The vanishing gradient problem is caused by the derivative of the activation function used
to create the neural network. The simplest solution to the problem is to replace the activation
function of the network. Instead of sigmoid, use an activation function such as ReLU

18. What is the problem with ReLU?


Key among the limitations of ReLU is the case where large weight updates can mean that
the summed input to the activation function is always negative, regardless of the input to the
network. This means that a node with this problem will forever output an activation value of
0.0. This is referred to as a “dying ReLU“

19. Why is ReLU used in deep learning?


The ReLU function is another non-linear activation function that has gained popularity in
the deep learning domain. ReLU stands for Rectified Linear Unit. The main advantage of
using the ReLU function over other activation functions is that it does not activate all the
neurons at the same time.

20. Why is ReLU better than Softmax?Give the equation of both ( April/May 2024)
As per our business requirement, we can choose our required activation function.
Generally , we use ReLU in hidden layer to avoid vanishing gradient problem and better
computation performance , and Softmax function use in last output layer .

3
Part – B

1. Draw the architecture of a single layer perceptron (SLP) and explain its operation.
Mention its advantages and disadvantages.(April/May 2024)
2. Draw the architecture of a Multilayer perceptron (MLP) and explain its operation.
Mention its advantages and disadvantages.
3. Explain the stochastic optimization methods for weight determination.
4. Explain the steps in the back propagation learning algorithm.What is importance of it in
designing neural networks (APRIL/MAY 2023)
5. Explain a deep feedforward network with a neat sketch (APRIL/MAY 2023)
6. Elaborate the process of training hideen layers by ReLU in deep networks ( Nov/Dec
2023)
7. Briefly explain hints and the different ways it can be used ( Nov/Dec 2023)
8. List the factors that affect the performance of multilayer feed-forward neural network.
9. Difference between a Shallow Net & Deep Learning Net.
10. How do you tune hyperparameters for better neural network performance? Explain in
detail.(April/May 2024)

Prepared By, Approved By,


Mrs. A.Deepika
Dr.R.Deepalakshmi
AP/AIDS
Prof& Head /AIDS

You might also like