Scribd
Upload
10 views

Week 7 - Lab

Uploaded by

saad.bashir1431
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
10 views

Week 7 - Lab

Uploaded by

saad.bashir1431
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

[Introduction to Fuzzy/Neural Systems]

7
[Back propagation algorithm]

Module 7 Exp 7 : on Back propagation


algorithm

Course Learning Outcomes:


C4. Conduct experiments using advanced tools related to fuzzy and neural
systems.
C5. Extend existing knowledge in designing a feedback fuzzy controller and
single/multilayer neural networks based on the given Industrial
requirements.
C7. Interpret and evaluate through written and oral forms.
C8. Lead multiple groups and projects with decision making responsibilities.
Topics
Exp 7 : on Back propagation algorithm

1. Aim/Objective:
What is the objective of this experiment?
2. Theory
Provide explanation on the perceptron learning rule
3. Procedure:
Step by step implementation of the implementation
4. Program
Python code to implement Perceptron learning rule
5. Result
State the your inputs and its corresponding output and figures
6. Conclusion
Summarize what you have learned from the experiment

Exp 7 : on Back propagation algorithm


Python code to implement back-propagation training algorithm
import numpy as np

def sigmoid(x):
return 1.0/(1.0 + np.exp(-x))
Course Module
def sigmoid_prime(x):
return sigmoid(x)*(1.0-sigmoid(x))

def tanh(x):
return np.tanh(x)

def tanh_prime(x):
return 1.0 - x**2

class NeuralNetwork:

def __init__(self, layers, activation='tanh'):


if activation == 'sigmoid':
self.activation = sigmoid
self.activation_prime = sigmoid_prime
elif activation == 'tanh':
self.activation = tanh
self.activation_prime = tanh_prime

# Set weights
self.weights = []
# layers = [2,2,1]
# range of weight values (-1,1)
# input and hidden layers - random((2+1, 2+1)) : 3 x 3
for i in range(1, len(layers) - 1):
r = 2*np.random.random((layers[i-1] + 1, layers[i] + 1)) -1
self.weights.append(r)
# output layer - random((2+1, 1)) : 3 x 1
r = 2*np.random.random( (layers[i] + 1, layers[i+1])) - 1
self.weights.append(r)

def fit(self, X, y, learning_rate=0.2, epochs=100000):


# Add column of ones to X
# This is to add the bias unit to the input layer
ones = np.atleast_2d(np.ones(X.shape[0]))
X = np.concatenate((ones.T, X), axis=1)

for k in range(epochs):
if k % 10000 == 0: print ('epochs:', k)

i = np.random.randint(X.shape[0])
a = [X[i]]

for l in range(len(self.weights)):
dot_value = np.dot(a[l], self.weights[l])
activation = self.activation(dot_value)
a.append(activation)
# output layer
error = y[i] - a[-1]
[Introduction to Fuzzy/Neural Systems]
7
[Back propagation algorithm]

deltas = [error * self.activation_prime(a[-1])]

# we need to begin at the second to last layer


# (a layer before the output layer)
for l in range(len(a) - 2, 0, -1):
deltas.append(deltas[-1].dot(self.weights[l].T)*self.activation_prime(a[l]))

# reverse
# [level3(output)->level2(hidden)] => [level2(hidden)->level3(output)]
deltas.reverse()

# backpropagation
# 1. Multiply its output delta and input activation
# to get the gradient of the weight.
# 2. Subtract a ratio (percentage) of the gradient from the weight.
for i in range(len(self.weights)):
layer = np.atleast_2d(a[i])
delta = np.atleast_2d(deltas[i])
self.weights[i] += learning_rate * layer.T.dot(delta)

def predict(self, x):


a = np.concatenate((np.array([[1]]), np.array([x])), axis=1)
for l in range(0, len(self.weights)):
a = self.activation(np.dot(a, self.weights[l]))
return a

if __name__ == '__main__':

nn = NeuralNetwork([2,2,1])

X = np.array([[0, 0],
[0, 1],
[1, 0],
[1, 1]])

y = np.array([0, 1, 1, 0])

nn.fit(X, y)

for e in X:
print(e,nn.predict(e))

Python code to implement back-propagation training algorithm using sigmoidal


activation function
Course Module
clc;
clear all;
close all;
%%Initialize the parameters
i1=1; i2=4;
i3=5;
n=3;
b1=1;b2=1;
t1=0.1; t2=0.05;
wb1=0.5; wb2=0.5;
w1=0.1; w2=0.2;
w3=0.3;w4=0.4;
w5=0.5;w6=0.6;
w7=0.7;w8=0.8;
w9=0.9;w10=0.1;
% alpha=(1/1+n)
alpha=0.01;
%% Hidden layer
h1net=sum(i1*w1+i2*w3+i3*w5+b1*wb1);
h1=(1/(1+exp(-h1net)));
h2net=sum(i1*w2+i2*w4+i3*w6+b1*wb1);
h2=(1/(1+exp(-h2net)));
%% output layer
o1net=sum(h1*w7+h2*w9+b2*wb2);
o1=(1/(1+exp(-o1net)));
o2net=sum(h1*w8+h2*w10+b2*wb2);
o2=(1/(1+exp(-o2net)));
%% Error
E1=1/2*((t1-o1)^2);
E2=1/2*((t2-o2)^2);
E=E1+E2;
%% Error derivates
change_in_error_1=-(t1-o1);
change_in_error_2=-(t2-o2);
change_in_o1net=o1*(1-o1);
change_in_o2net=o2*(1-o2);
change_in_h1=change_in_error_1*change_in_o1net*w7+change_in_error_2*change_in_o2ne
t*w8;
change_in_h2=change_in_error_1*change_in_o1net*w9+change_in_error_2*change_in_o2ne
t*w10;
change_in_h1net=h1*(1-h1);
change_in_h2net=h2*(1-h2);
change_in_net_w7=h1*w7^(1-1);
change_in_net_w8=h1*w8^(1-1);
change_in_net_w9=h2*w9^(1-1);
change_in_net_w10=h2*w9^(1-1);
change_in_net_wb2=b2*wb2^(1-1);
change_in_net_w1=i1*w1^(1-1);
change_in_net_w2=i1*w2^(1-1);
change_in_net_w3=i2*w3^(1-1);
change_in_net_w4=i2*w4^(1-1);
[Introduction to Fuzzy/Neural Systems]
7
[Back propagation algorithm]

change_in_net_w5=i3*w5^(1-1);
change_in_net_w6=i3*w6^(1-1);
change_in_net_wb1=b1*wb1^(1-1);
%% gradient descent Input to Hidden layer
G1=change_in_h1*change_in_h1net*change_in_net_w1;
G2=change_in_h2*change_in_h2net*change_in_net_w2;
G3=change_in_h1*change_in_h1net*change_in_net_w3;
G4=change_in_h2*change_in_h2net*change_in_net_w4;
G5=change_in_h1*change_in_h1net*change_in_net_w5;
G6=change_in_h2*change_in_h2net*change_in_net_w6;
Gb1=change_in_error_1*change_in_o1net*w7*change_in_h1net*change_in_net_wb1+chang
e_in_error_2*change_in_o2net*w10*change_in_h2net*change_in_net_wb2;

%%%% gradient descent hidden to output layer


G7=change_in_error_1*change_in_o1net*change_in_net_w7;
G8=change_in_error_2*change_in_o2net*change_in_net_w8;
G9=change_in_error_1*change_in_o1net*change_in_net_w9;
G10=change_in_error_2*change_in_o2net*change_in_net_w10;
Gb2=change_in_error_2*change_in_o2net*change_in_net_wb2+change_in_error_1*change_i
n_o1net*change_in_net_wb2;
%% Updated weights
updated_W1=w1-alpha*G1;
updated_W2=w2-alpha*G2;
updated_W3=w3-alpha*G3;
updated_W4=w4-alpha*G4;
updated_W5=w5-alpha*G5;
updated_W6=w6-alpha*G6;
updated_W7=w7-alpha*G7;
updated_W8=w8-alpha*G8;
updated_W9=w9-alpha*G9;
updated_W10=w10-alpha*G10;
updated_Wb1=wb1-alpha*Gb1;
updated_Wb2=wb2-alpha*Gb2;
%%

<Exercise 1.
Lab Activity: Simulation
Design and develop the neural network system for the following experiment
Experiment 1: Back Propagation algorithm
1. Design and train a neural network system based on back propagation
algorithm using hyperbolic sigmoidal activation function.
2. Tune the neural network model and minimize the error by updating the
weights and perform the testing.

Course Module
3. Run the simulation in group and explain the working principles of the
algorithm.
4. Interpret the output of the designed neural network system by varying the
inputs.

References and Supplementary Materials


Books and Journals

1. Van Rossum, G. (2007). Python Programming Language. In USENIX annual technical


conference (Vol. 41, p. 36).
2. SN, S. (2003). Introduction to artificial neural networks.
3. Rashid, T. (2016). Make your own neural network. CreateSpace Independent Publishing
Platform.

Online Supplementary Reading Materials

1. Chaitanya Singh; How to Install Python. (n.d.). Retrieved 14 May 2020, from
https://beginnersbook.com/2018/01/python-installation/; 14-05-2020

You might also like