Scribd
Upload
15 views

Exercise_Machine Learning_2

Uploaded by

datucha.imnaishvili
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
15 views

Exercise_Machine Learning_2

Uploaded by

datucha.imnaishvili
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

3 Neural Network Representations

You are given a number of functions (a-h) of a single variable, x, which are graphed below. The computation
graphs on the following pages will start o↵ simple and get more complex, building up to neural networks. For
each computation graph, indicate which of the functions below they are able to represent.

( (
(a) 2x (b) 4x 5 2x 5 x 2.5 2x 5 x 2.5
(c) (d)
0 x < 2.5 0 x> 2.5

( 8 8
x+3 x 2 >
<3 x0 (g) log(x) >
<0.5x x0
(e)
1 x<2 (f) 3 x 0<x3 (h) 0 0<x3
>
: >
:
0 x>3 3x 9 x>3

1. Consider the following computation graph, computing a linear transformation with scalar input x, weight
w, and output o, such that o = wx. Which of the funcions can be represented by this graph? For the
options which can, write out the appropriate value of w.

4
2. Now we introduce a bias term b into the graph, such that o = wx + b (this is known as an affine function).
Which of the functions can be represented by this network? For the options which can, write out an
appropriate value of w, b.

3. We can introduce a non-linearity into the network as indicated below. We use the ReLU non-linearity,
which has the form ReLU (x) = max(0, x). Now which of the functions can be represented by this neural
network with weight w and bias b? For the options which can, write out an appropriate value of w, b.

4. Now we consider neural networks with multiple affine transformations, as indicated below. We now have
two sets of weights and biases w1 , b1 and w2 , b2 . We denote the result of the first transformation h such
that h = w1 x + b1 , and o = w2 h + b2 . Which of the functions can be represented by this network? For
the options which can, write out appropriate values of w1 , w2 , b1 , b2 .

5
5. Next we add a ReLU non-linearity to the network after the first affine transformation, creating a hidden
layer. Which of the functions can be represented by this network? For the options which can, write out
appropriate values of w1 , w2 , b1 , b2 .

6. Now we add another hidden layer to the network, as indicated below. Which of the functions can be
represented by this network?

6
7. We’d like to consider using a neural net with just one hidden layer, but have it be larger – a hidden layer
of size 2. Let’s first consider using just two affine functions, with no nonlinearity in between. Which of
the functions can be represented by this network?

8. Now we’ll add a non-linearity between the two affine layers, to produce the neural network below with a
hidden layer of size 2. Which of the functions can be represented by this network?

You might also like