AALIM MUHAMMED SALEGH COLLEGE OF ENGINEERING

DEPARTMENT OF INFORMATION TECHNOLOGY

L T P C
CCS355 NEURAL NETWORKS AND DEEP LEARNING
2 0 2 3

Focused Questions and Answers

1. What is a neural network?

Neural networks, also known as artificial neural networks (ANNs) or simulated neural networks
(SNNs), are a subset of machine learning and are at the heart of deep learning algorithms. Their
name and structure are inspired by the human brain, mimicking the way that biological neurons
signal to one another.

2. Why are neural networks important?

Neural networks can help computers make intelligent decisions with limited human assistance.
This is because they can learn and model the relationships between input and output data that are
nonlinear and complex. For instance, they can do the following tasks.

3. What are neural networks used for?

Neural networks have several use cases across many industries, such as the following:
 Medical diagnosis by medical image classification
 Targeted marketing by social network filtering and behavioral data analysis
 Financial predictions by processing historical data of financial instruments
 Electrical load and energy demand forecasting
 Process and quality control
 Chemical compound identification

4. List the Type of neural network.

(i) ANN– It is also known as an artificial neural network

(ii) CNN– It is also known as Convolutional Neural Networks.
(iii) RNN– It is also known as Recurrent Neural Networks.

5. What are the three types of learning in Neural Network?

1. Supervised Learning
2. Unsupervised Learning
3. Reinforcement Learning

6. What are the different types of ANN?

 Feed forward artificial neural networks

1
  Perceptron and Multilayer Perceptron neural networks
 Radial basis function artificial neural networks
 Recurrent neural networks
 Modular neural networks

7. What are the important terminologies in ANN?

 Weights
 Bias
 Threshold
 Learning Rate
 Target value
 Error

 Weights: each neuron is linked to the other neurons through connection links that carry
weight. The weight has information and data about the input signal. The output depends
solely on the weights and input signal. The weights can be presented in a matrix form
that is known as the Connection matrix.
 Bias: Bias is a constant that is added to the product of inputs and weights to calculate
the product. It is used to shift the result to the positive or negative side. The net input
weight is increased by a positive bias while The net input weight is decreased by a
negative bias.
 Bias: Bias is a constant that is added to the product of inputs and weights to calculate
the product. It is used to shift the result to the positive or negative side. The net input
weight is increased by a positive bias while The net input weight is decreased by a
negative bias.
 Learning Rate: The learning rate is denoted α. It ranges from 0 to 1. It is used for
balancing weights during the learning of ANN.
 Target value: Target values are Correct values of the output variable and are also
known as just targets.
 Error: It is the inaccuracy of predicted output values compared to Target Values.

8. What is supervised learning?

Basically supervised learning is when we teach or train the machine using data that is well-
labelled. Which means some data is already tagged with the correct answer. After that, the
machine is provided with a new set of examples(data) so that the supervised learning algorithm
analyses the training data(set of training examples) and produces a correct outcome from
labeled data.

9. What are the classifications of supervised learning?

 Classification: A classification problem is when the output variable is a category, such

as “Red” or “blue” , “disease” or “no disease”.
2
  Regression: A regression problem is when the output variable is a real value, such as
“dollars” or “weight”.

10. What is learning algorithm?

A prescribed set of well-defined rules for the solution of a learning problem is called a
learning algorithm.

11. How could a neural network learn?

1.Developing new connections. 2. Deleting existing connections. 3. Changing connecting
weights, (and practically). 4. Changing the threshold values of neurons, (and practically). 5.
Changing activation function, propagation function or output function. 6. Developing new
neurons. 7. Deleting existing neurons.

12. What is Perceptron?

A Perceptron is the smallest element of a neural network. Perceptron is a single-layer neural
network linear or a Machine Learning algorithm used for supervised learning of various binary
classifiers. It works as an artificial neuron to perform computations by learning elements and
processing them for detecting the business intelligence and capabilities of the input data.

13. What are the basic components of Perceptron?

The basic components of a Perceptron are:

1. Input Layer: The input layer consists of one or more input neurons, which receive input
signals from the external world or from other layers of the neural network.

2. Weights: Each input neuron is associated with a weight, which represents the strength of
the connection between the input neuron and the output neuron.
3. Bias: A bias term is added to the input layer to provide the Perceptron with additional
flexibility in modeling complex patterns in the input data.
4. Activation Function: The activation function determines the output of the Perceptron
based on the weighted sum of the inputs and the bias term. Common activation functions used
in Perceptrons include the step function, sigmoid function, and ReLU function.
5. Output: The output of the Perceptron is a single binary value, either 0 or 1, which
indicates the class or category to which the input data belongs.
6. Training Algorithm: The Perceptron is typically trained using a supervised learning
algorithm such as the Perceptron learning algorithm or back propagation. During training, the
weights and biases of the Perceptron are adjusted to minimize the error between the predicted
output and the true output for a given set of training examples.

14. What is an activation function and why use them?

3
The activation function decides whether a neuron should be activated or not by calculating the
weighted sum and further adding bias to it. The purpose of the activation function is to
introduce non-linearity into the output of a neuron.

15. What are the types of activation function?

1) Linear Activation Functions

2) Non-Linear Activation Functions
a) Sigmoid Activation Functions
b) Tanh Activation Functions
c) ReLU Activation Functions
d) Maxout:
e) ELU
f) Softmax Activation Functions

Part B
1. Explain briefly about the operation of biological neural network with neat sketch
2. Discuss Supervised learning and Unsupervised learning.
3. Explain the evolution of Neural Network.
4. Explain briefly about artificial neural network.
5. Explain in detail supervised learning network.

4
 UNIT II ASSOCIATIVE MEMORY AND UNSUPERVISED LEARNING
NETWORKS

1. What is pattern association in neural network?

Pattern association is the process of memorizing input-output patterns in a hetero associative

network architecture, or input patterns only in an autoassociative network, in order to recall the
patterns when a new input pattern is presented.

2. What is Associative Memory Network?

An associative memory network can store a set of patterns as memories. When the associative
memory is being presented with a key pattern, it responds by producing one of the stored
patterns, which closely resembles or relates to the key pattern.

3. What are Training Algorithms for Pattern Association?

There are two algorithms developed for training of pattern association nets.

1. Hebb Rule
2. Outer Products Rule

4. What is Hebb Rule?

The Hebb rule is widely used for finding the weights of an associative memory neural network.
The training vector pairs here are denoted as s:t. The weights are updated until there is no weight
change.

5. What is Outer Products Rule?

Outer products rule is a method for finding weights of an associative net.

Input=> s = (s1, ... ,si, ... ,sn)
Output=> t= (t1, ... ,tj, ... ,tm)
The outer product of the two vectors is the product of the matrices S = s T and T = t, i.e., between
[n X 1] matrix and [1 x m] matrix. The transpose is to be taken for the input matrix given.

6. What are the two types of associative memories?

 Auto Associative Memory

 Hetero Associative memory
5
7. What is hetero associative memory?

In a hetero-associate memory, the training input and the target output vectors are
different. The weights are determined in a way that the network can store a set of pattern
associations. The association here is a pair of training input target output vector pairs
(s(p), t(p)), with p = 1,2,…p. Each vector s(p) has n components and each vector t(p) has
m components. The determination of weights is done either by using Hebb rule or delta
rule. The net finds an appropriate output vector, which corresponds to an input vector x,
that may be either one of the stored patterns or a new pattern.

8. What is BAM?

Bidirectional associative memory - This is hetero-associative memory, for an input

pattern, it returns another pattern which is potentially of a different size.

9. What are the types of BAM?

Two Forms of BAM – (1) Discrete BAM (2) Continuous BAM

10. What is Fixed Weight Competitive Nets?

During training process also the weights remains fixed in these competitive networks. The idea
of competition is used among neurons for enhancement of contrast in their activation functions.
In this, two networks- Maxnet and Hamming networks.

11. What is Self organizing maps?

Self Organizing Map (or Kohonen Map or SOM) is a type of Artificial Neural Network which
is also inspired by biological models of neural systems from the 1970s. It follows an
unsupervised learning approach and trained its network through a competitive learning
algorithm. SOM is used for clustering and mapping (or dimensionality reduction) techniques to
map multidimensional data onto lower-dimensional which allows people to reduce complex
problems for easy interpretation. SOM has two layers, one is the Input layer and the other one is
the Output layer.

Part B

1. Explain in detail about Autoassociative Memory Network.

2. Explain Training Algorithms for Pattern Association.
3. Explain in detail about Bidirectional Associative Memory.
4. Explain briefly about Iterative Autoassociative Memory Networks.

5. Discuss about Self-Organizing Feature Maps.

6
UNIT III THIRD-GENERATION NEURAL NETWORKS

1. What is Spiking Neural Network (SNN)?

Artificial neural networks that closely mimic natural neural networks are known as spiking
neural networks (SNNs). In addition to neuronal and synaptic status, SNNs incorporate time into
their working model. The idea is that neurons in the SNN do not transmit information at the end
of each propagation cycle (as they do in traditional multi-layer Perceptron networks), but only
when a membrane potential – a neuron’s intrinsic quality related to its membrane electrical
charge – reaches a certain value, known as the threshold.

2. What is a convolutional neural network?

A convolutional neural network is a specific kind of neural network with multiple layers. It
processes data that has a grid-like arrangement then extracts important features. One huge
advantage of using CNNs is that you don't need to do a lot of pre-processing on images.

3. What are the different types of CNN?

Different types of CNNs

1D CNN: With these, the CNN kernel moves in one direction. 1D CNNs are usually used on
time-series data.
2D CNN: These kinds of CNN kernels move in two directions. You'll see these used with image
labelling and processing.
3D CNN: This kind of CNN has a kernel that moves in three directions. With this type of CNN,
researchers use them on 3D images like CT scans and MRIs.

4. What is deep neural network?

A deep neural network (DNN) is an ANN with multiple hidden layers between the input and
output layers. Similar to shallow ANNs, DNNs can model complex non-linear relationships.

The main purpose of a neural network is to receive a set of inputs, perform progressively
complex calculations on them, and give output to solve real world problems like classification.
We restrict ourselves to feed forward neural networks.

5. What is Deep Belief Networks

A deep belief network is a class of Deep Neural Network that comprises of multi-layer belief
networks.
Steps to perform DBN:
1. With the help of the Contrastive Divergence algorithm, a layer of features is learned from
perceptible units.
2. Next, the formerly trained features are treated as visible units, which perform learning of
features.
7
 3. Lastly, when the learning of the final hidden layer is accomplished, then the whole DBN
is trained.

6. What is ELM?
ELM (Extreme Learning Machines) are feed forward neural networks. The ELMs are believed to
have the ability to learn thousands of times faster than networks trained using the back
propagation technique.

7. What is Computer Vision (CV)?

Computer vision is an area of machine learning dedicated to interpreting and understanding
images and video. It is used to help teach computers to “see” and to use visual information to
perform visual tasks that humans can.

Part B
1. Explain in detail about Convolutional neural network.
2. Discuss about various convolution operation.
3. Explain Efficient Convolution Algorithms
4. Discuss about computer vision.
5.Explain image compression using CNN

8
UNIT IV DEEP FEED FORWARD NETWORKS

1. What is a feed forward neural network?

Feed forward neural networks are artificial neural networks in which nodes do not form loops.
This type of neural network is also known as a multi-layer neural network as all information is
only passed forward.
During data flow, input nodes receive data, which travel through hidden layers, and exit output
nodes. No links exist in the network that could get used to by sending information back from the
output node.

2. What are the layers of feed forward neural network?

Input Layer, Output Layer, Hidden Layer

3. What is deep learning?

Deep learning is a subset of machine learning, which is essentially a neural network with three or
more layers. These neural networks attempt to simulate the behavior of the human brain—albeit
far from matching its ability—allowing it to “learn” from large amounts of data. While a neural
network with a single layer can still make approximate predictions, additional hidden layers can
help to optimize and refine for accuracy.

4. What is probabilistic deep learning?

Probabilistic deep learning is deep learning that accounts for uncertainty, both model
uncertainty and data uncertainty. It is based on the use of probabilistic models and deep
neural networks. We distinguish two approaches to probabilistic deep learning: probabilistic
neural networks and deep probabilistic models.

5. What is gradient descent?

Gradient descent (GD) is an iterative first-order optimization algorithm used to find a local
minimum/maximum of a given function. This method is commonly used in machine learning
(ML) and deep learning(DL) to minimize a cost/loss function (e.g. in a linear regression).

6. What is chain rule?

Chain Rule

Suppose we have a 3D surface given by z = f(x,y). If x and y are parametrized by some

independent variable t, then we will actually obtain a curve in space. According to the chain rule,
the rate of change of f with t can be calculated as follows:

9
7. What is back propagation in deep neural network?
Back propagation is one of the important concepts of a neural network. For a single training
example, Back propagation algorithm calculates the gradient of the error function. Back
propagation can be written as a function of the neural network. Back propagation algorithms are
a set of methods used to efficiently train artificial neural networks following a gradient descent
approach which exploits the chain rule.

8. What is Regularization?
Regularization is a technique used in machine learning and deep learning to prevent overfitting
and improve the generalization performance of a model. It involves adding a penalty term to the
loss function during training.

9. What is Data Augmentation?

Data augmentation is a technique of artificially increasing the training set by creating modified
copies of a dataset using existing data. It includes making minor changes to the dataset or using
deep learning to generate new data points.

10. Augmented vs. Synthetic data

Augmented data is driven from original data with some minor changes. In the case of image
augmentation, we make geometric and color space transformations (flipping, resizing, cropping,
and brightness, contrast) to increase the size and diversity of the training set.

Synthetic data is generated artificially without using the original dataset. It often uses DNNs
(Deep Neural Networks) and GANs (Generative Adversarial Networks) to generate synthetic
data.

11. What Is Bagging?

Bagging, also known as Bootstrap aggregating, is an ensemble learning technique that helps to
improve the performance and accuracy of machine learning algorithms. It is used to deal with
bias-variance trade-offs and reduces the variance of a prediction model. Bagging avoids
overfitting of data and is used for both regression and classification models, specifically for
decision tree algorithms.

Part B
1. Explain the Probabilistic Theory of Deep Learning.
2. Explain in detail about Chain Rule and Backpropagation.
3. Explain in detail about Data augmentation.

10
UNIT V RECURRENT NEURAL NETWORKS

1. What is Recurrent Neural Network (RNN)?

Recurrent Neural Network(RNN) is a type of Neural Network where the output from the
previous step is fed as input to the current step. In traditional neural networks, all the inputs and
outputs are independent of each other, but in cases when it is required to predict the next word of
a sentence, the previous words are required and hence there is a need to remember the previous
words.

2. What Is a Recursive Neural Network?

Recursive Neural Networks (RvNNs) are a class of deep neural networks that can learn detailed
and structured information. With RvNN, you can get a structured prediction by recursively
applying the same set of weights on structured inputs. The word recursive indicates that the
neural network is applied to its output.

3. What are the types of RNN?

The four commonly used types of Recurrent Neural Networks are:
1. One-to-One
2. One-to-Many
3. Many-to-One
4. Many-to-Many

4. What are deep recurrent networks?

Deep RNN (Recurrent Neural Network) refers to a neural network architecture that has multiple
layers of recurrent units. Recurrent Neural Networks are a type of neural network that is
designed to handle sequential data, such as time series or natural language, by maintaining an
internal memory of previous inputs.

5. What are the applications of Recurrent Neural Networks:

 Prediction problems.
 Machine Translation.
 Speech Recognition.
 Language Modelling and Generating Text.
 Video Tagging.
 Generating Image Descriptions.
 Text Summarization.
 Call Center Analysis.
11
6. What is LSTM in deep learning?
Long Short-Term Memory (LSTM) is a type of Recurrent Neural Network (RNN) that is
specifically designed to handle sequential data, such as time series, speech, and text.

7. What is an Autoencoder?
Autoencoders are neural networks designed to learn a low-dimensional representation of a given
input. Autoencoders typically consist of two components: an encoder which learns to map input
data to a lower dimensional representation and a decoder, which learns to map the representation
back to the input data.

8. What is encoder and decoder?

The encoder-decoder architecture for recurrent neural networks is the standard neural machine
translation method that rivals and in some cases outperforms classical statistical machine
translation methods.

Part B
1. Explain in detail about RNN.
2. Explain Bidirectional RNNs
3. Explain Deep Recurrent Networks
4. Discuss about NLP
5. Discuss about encoder and decoder.

12