NPTEL Online Certification Courses

Indian Institute of Technology Kharagpur

Deep Learning
Assignment- Week 4
TYPE OF QUESTION: MCQ/MSQ
Number of questions: 10 Total mark: 10 X 1 = 10
______________________________________________________________________________

QUESTION 1:
A given cost function is of the form J(θ) = θ2 - θ+2? What is the weight update rule for gradient
descent optimization at step t+1? Consider, 𝛼=0.01 to be the learning rate.

a. 𝜃𝑡+1 = 𝜃𝑡 − 0.01(2𝜃 − 1)
b. 𝜃𝑡+1 = 𝜃𝑡 + 0.01(2𝜃)
c. 𝜃𝑡+1 = 𝜃𝑡 − (2𝜃 − 1)
d. 𝜃𝑡+1 = 𝜃𝑡 − 0.01(𝜃 − 1)

Correct Answer: a

Detailed Solution:

𝜕𝐽(𝜃)
= 2𝜃 − 1
𝜕𝜃
So, weight update will be
𝜃𝑡+1 = 𝜃𝑡 − 0.01(2𝜃 − 1)
______________________________________________________________________________

QUESTION 2:
Can you identify in which of the following graph gradient descent will not work correctly?

a. First figure
b. Second figure
c. First and second figure
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

d. Fourth figure
Correct Answer: b

Detailed Solution:

This is a classic example of saddle point problem of gradient descent. In the second graph
gradient descent may get stuck in the saddle point.
______________________________________________________________________________

QUESTION 3:
From the following two figures can you identify which one corresponds to batch gradient
descent and which one to Stochastic gradient descent?

a. Graph-A: Batch gradient descent, Graph-B: Stochastic gradient descent

b. Graph-B: Batch gradient descent, Graph-A: Stochastic gradient descent
c. Graph-A: Batch gradient descent, Graph-B: Not Stochastic gradient descent
d. Graph-A: Not batch gradient descent, Graph-B: Not Stochastic gradient descent

Correct Answer: a

Detailed Solution:

The graph of cost vs epochs is quite smooth for batch gradient descent because we are
averaging over all the gradients of training data for a single step. The average cost over the
epochs in Stochastic gradient descent fluctuates because we are using one example at a
time.

______________________________________________________________________________
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

QUESTION 4:
Suppose for a cost function 𝐽(𝜃) = 0.25𝜃 2 as shown in graph below, in which point do you feel
magnitude of weight update will be more? 𝜃 is plotted along horizontal axis.

a. Red point (Point 1)

b. Green point (Point 2)
c. Yellow point (Point 3)
d. Red (Point 1) and yellow (Point 3) have same magnitude of weight update

Correct Answer: a

Detailed Solution:

Weight update is directly proportional to the magnitude of the gradient of the cost
𝜕𝐽(𝜃)
function. In our case, 𝜕𝜃
= 0.5𝜃. So, the weight update will be more for higher values of 𝜃.

______________________________________________________________________________
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

QUESTION 5:
Which of the following options closely relate to the following graph? Green cross are the
samples of Class-A while mustard rings are samples of Class-B and the red line is the separating
line between the two class.

a. Overfitting
b. Underfitting
c. Appropriate fit
d. Cannot comment

Correct Answer: b

Detailed Solution:

The figure given in the question is an example of under fitting as the separating line is not
sufficient to explain the variance.

____________________________________________________________________________

QUESTION 6:
Among the following options which can be used to curb the problem of overfitting?

a. Regularization
b. Training the network for longer time
c. Introducing more complex model architecture
d. Modifying the cost function that enhances the weights of the model parameters
by a constant value

Correct Answer: a

Detailed Solution:

Regularization basically adds the penalty as model complexity increases. Regularization

parameter (lambda) penalizes all the parameters except intercept so that model generalizes
the data and won’t overfit.
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

____________________________________________________________________________

QUESTION 7:
Which among the following options give the range for a logistic function?

a. -1 to 1
b. -1 to 0
c. 0 to 1
d. 0 to infinity

Correct Answer: c

Detailed Solution:

Refer to lectures, specifically the formula for logistic function.

______________________________________________________________________________

QUESTION 8:
Consider, a dataset having 100 training example. For ith sample can you identify the cost
function for linear regression? 𝑦𝑝𝑖 is the predicted output for ith sample, yi.
1
a. ∑100 𝑖 𝑖 2
𝑖=1(𝑦𝑝 − 𝑦 )
200
1
b. ∑200 𝑖 𝑖 2
𝑖=1(𝑦𝑝 − 𝑦 )
100
1
c. ∑200 𝑖 𝑖 2
𝑖=1(𝑦𝑝 − 𝑦 )
200
1
d. ∑100 𝑖 𝑖 3
𝑖=1(𝑦𝑝 − 𝑦 )
200

Correct Answer: a

Detailed Solution:
𝟏
Linear regression cost function is given by 𝟐𝑴 ∑𝑴 𝒊 𝒊 𝟐
𝒊=𝟏(𝒚𝒑 − 𝒚 ) . Here M is the number of
training examples.

______________________________________________________________________________
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

QUESTION 9:
For a XNOR function as given in the figure below, activation function of each node is given by:
1, 𝑥 ≥ 0
𝑓(𝑥) = { . Consider 𝑋1 = 0 and 𝑋2 = 1, what will be the output for the above
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
neural network?

a. 1.5
b. 2
c. 0
d. 1

Correct Answer: c

Detailed Solution:

Output of 𝒂𝟏 : 𝒇(𝟎. 𝟓 ∗ 𝟏 + −𝟏 ∗ 𝟎 + −𝟏 ∗ 𝟏) = 𝒇(−𝟎. 𝟓) = 𝟎

Output of 𝒂𝟐 : 𝒇(−𝟏. 𝟓 ∗ 𝟏 + 𝟏 ∗ 𝟎 + 𝟏 ∗ 𝟏) = 𝒇(−𝟎. 𝟓) = 𝟎

Output of 𝒂𝟑 : 𝒇(−𝟎. 𝟓 ∗ 𝟏 + 𝟏 ∗ 𝟎 + 𝟏 ∗ 𝟎) = 𝒇(−𝟎. 𝟓) = 𝟎

So, the correct answer is c.

____________________________________________________________________________

QUESTION 10:
An artificial neuron receives n inputs 𝑥1 , 𝑥2 , 𝑥3 , … . . 𝑥𝑛 with weights 𝑤1 , 𝑤2 , 𝑤3 , … . . 𝑤𝑛 attached
to the input links. The weighted sum_________________ is computed to be passed on to a
non-linear filter Φ called activation function to release the output. Fill in the blanks by choosing
one option from the following.

a. ∑𝑖 𝑤𝑖
b. ∑𝑖 𝑥𝑖
c. ∑𝑖 𝑤𝑖 + ∑𝑖 𝑥𝑖
d. ∑𝑖 𝑤𝑖 𝑥𝑖
 NPTEL Online Certification Courses
Indian Institute of Technology Kharagpur

Correct Answer: d

Detailed Solution:

Refer to the lecture.

______________________________________________________________________

______________________________________________________________________________

************END*******