Scribd
Upload
4 views7 pages

Practice QuestionsV1

The document discusses various topics related to multi-class perceptrons, including weight updates, gradient descent, binary cross-entropy loss calculations, softmax function properties, and cross-entropy loss for classification problems. It provides specific examples and calculations for updating weights and computing losses in neural networks. Additionally, it touches on the application of stochastic gradient descent (SGD) and momentum in optimization processes.

Uploaded by

shaahidndsm
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
4 views7 pages

Practice QuestionsV1

The document discusses various topics related to multi-class perceptrons, including weight updates, gradient descent, binary cross-entropy loss calculations, softmax function properties, and cross-entropy loss for classification problems. It provides specific examples and calculations for updating weights and computing losses in neural networks. Additionally, it touches on the application of stochastic gradient descent (SGD) and momentum in optimization processes.

Uploaded by

shaahidndsm
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 7

The current weight vectors for a multi-class perceptron are:

W1=[2,−6,9],W2=[3,−5,9],W3=[9,9,−3].
Update the weight vectors with a new training instance with f(x) = (−3,2,5),y*=2.
What are the weight vectors W1, W2 and W3 after the update?

**************************************************************
**************************************************************************

Given this Perceptron with weights as shown

What would the new weight of w_2 be?


*************************************************************************
*******************************************************************

Apply the gradient descent algorithm for the following function for 3 iterations assuming
learning rate alpha 0.01. Let's assume the starting values are x = 0 and y = 0 .

*************************************************************************
Given the following predictions and true labels, calculate the binary cross-entropy loss. (Use
the natural logarithm (log x := ln x) in the calculation of the BCE.
Predictions: [0.7, 0.1, 0.8, and 0.2] True labels: [1, 0, 0, and 1]

*************************************************************************
**************************************************************************

a) Show that the output of the softmax function with any input is a valid probability
distribution.

Now, consider a neural network classifier that outputs raw scores (also known as logits) for
three classes. For a particular input, the raw scores output by the network are given by the
vector z=2,1,-1, please answer the following questions

b) Calculate the softmax probabilities for each class.

c) If the true class for the input is the second class, calculate the cross-entropy loss.

***********************************************************************
Calculate the loss using SoftMax Cost Function considering the following details.
scores =3.2,5.1,-1.7
label =1,0,0

*************************************************************************
The hyperbolic tangent activation function is defined by y=tanh(x)=[exp(x)-exp(-x)]/[exp(x)
+exp(-x)]. The derivative (slope) at x=0 is equal to?

*****************************************************************

*******************************************************************
*************************************************************************

************************************************************************
Consider a multi-class classification problem with four classes, A, B, C, and D. To solve the
problem, you train a multi-class classifier, which outputs the probabilities of each class.
Consider a data example, whose true class is B, and the probabilities produced by the
classifier when applied to this data example is P(A) = 0.1, P(B) = 0.5, P(C) = 0.2 and
P(D) = 0.2. What is the cross entropy loss for this data example?
Apply SGD for the given data dataset for two iterations with the LR is = 0.05 and loss
function as L2 norm. Consider initial bias as 0. The target function is given in the table
6w3*x +8
y = w1*x1^2 + w2*x2^2 - 3
Learning
Rate 0.05 Loss Function SSR

w1 -0.064 x1 x2 x3 y
w2 -0.012 5 1 2 12
w3 -0.041 2 4 1 -14
3 2 2 6
3 6 3 -6
3 4 4 -8
7 4 5 -9

Consider a function

Apply the SGD with momentum and using the given parameters with x 0 = 3.12, update the
value of x for three iterations.
b0 0 μ = 0.40
lr = 0.06

*************************************************************************

You might also like