7-Working Example-01-08-2024
Uploaded by
vemuripraveena26227-Working Example-01-08-2024
Uploaded by
vemuripraveena2622Introduction to Neural
Networks
Example 1: whether you should go surfing (Yes:
1, No: 0).
• The decision to go or not to go is our predicted outcome, or y-hat.
• Let’s assume that there are three factors influencing your decision-
making:
• Are the waves good? (Yes: 1, No: 0)
• Is the line-up empty? (Yes: 1, No: 0)
• Has there been a recent shark attack? (Yes: 0, No: 1)
• X1 = 1, since the waves are pumping
• X2 = 0, since the crowds are out
• X3 = 1, since there hasn’t been a recent shark attack
Assign some weights to determine importance. Larger weights
signify that particular variables are of greater importance to the
decision or outcome.
• W1 = 5, since large swells don’t come around often
• W2 = 2, since you’re used to the crowds
• W3 = 4, since you have a fear of sharks
• Finally, we’ll also assume a threshold value of 3, which would translate
to a bias value of 3.
• With all the various inputs, we can start to plug in values into the
formula to get the desired output.
• Y-hat = (1*5) + (0*2) + (1*4) + (-3) = 6 6 > 1
Summary
• our brain is fully aware of the context of the data we use in making
predictions
• In the Neural Network nothing other than apply the formula to the
bunch of data to produce a probability/guess that we term as
prediction.
• While our brain reasons with the data it comes in contact with,
• the Neural Network makes no sense out of the data, it rather takes in
input, applies a formula and produces an output –
Example 2:
Forward Propagation
1
6
Delhi Weighted Sum
1 2
9
0 /1
0
8
Shimla
1
A neural network is composed of 3 types
of layers
• Input layer — It is used to pass in our input(an image, text or any
suitable type of data for NN).
• Hidden Layer — These are the layers in between the input and output
layers. These layers are responsible for learning the mapping between
input and output.
• Output Layer — This layer is responsible for giving us the output of
the NN given our inputs.
Example-2
• Example 3:
• 1 input layer with 2 inputs
(X1 and X2),
• 1 output layer with 1 output.
There are no hidden layers.
• The weights of the inputs are
W1 and W2, respectively.
• The bias is treated as a new
input neuron to the output
neuron which has a fixed
value +1 and a weight b. Both
the weights and biases could
be referred to as parameters.
• s is the sum of products (SOP) between each input and its corresponding
weight:
• s=X1* W1+ X2*W2+b
Forward pass
• s=X1* W1+ X2*W2+b
• s=0.1* 0.5+ 0.3*0.2+1.83
• s=1.94
• The value 1.94 is then applied to the activation function (sigmoid),
which results in the value 0.874352143.
• Actual output : 0.03
• Predicted output : 0.874352143
• It’s obvious that there’s a difference between the desired and expected
output. But why?
Why backpropagation????
• The backpropagation algorithm is one of the algorithms responsible
for updating network weights with the objective of reducing the
network error. It’s quite important.
• If the current error is high, the network didn’t learn properly from the
data.
• What does this mean? It means that the current set of weights isn’t
accurate enough to reduce the network error and make accurate
predictions.
• As a result, we should update network weights to reduce the network
error.
• s is the sum of products (SOP) between each input and its corresponding
weight:
• s=X1* W1+ X2*W2+b
Forward pass
• s=X1* W1+ X2*W2+b
• s=0.1* 0.5+ 0.3*0.2+1.83
• s=1.94
• The value 1.94 is then applied to the activation function (sigmoid),
which results in the value 0.874352143.
Error of our network based on an error
function
• The error functions tell how close the predicted output(s) are from the
desired output(s).
• The optimal value for error is zero, meaning there’s no error at all, and
both desired and predicted results are identical.
• One of the error functions is the squared error function
• Knowing that there’s an error, what should we do? We should minimize
it.
• To minimize network error, we must change something in the network.
• Remember that the only parameters we can change are the weights and
biases.
• We can try different weights and biases, and then test our network.
• We calculate the error, then the forward pass ends, and we should start
the backward pass to calculate the derivatives and update the
parameters.
• The parameters-update equation just depends on the learning rate
to update the parameters. It changes all the parameters in a direction
opposite to the error.
• But, using the backpropagation algorithm, we can know how each
single weight correlates with the error. This tells us the effect of each
weight on the prediction error. That is, which parameters do we
increase, and which ones do we decrease to get the smallest prediction
error?
• For example, the backpropagation algorithm could tell us useful
information, like that increasing the current value of W1 by 1.0
increases the network error by 0.07. This shows us that a smaller value
for W1 is better to minimize the error.
• One important operation used in the backward pass is to calculate
derivatives
• Y=x^2Z+H
• our target is to calculate ∂E/W1 and ∂E/W2 as we have just two
weights W1 and W2. Let’s calculate them.
