[Introduction to Fuzzy/Neural Systems]

2
[On McCulloch-Pitts Neuron Model]

Module 2 Exp 2: On McCulloch-Pitts Neuron

Model

Course Learning Outcomes:

C4. Conduct experiments using advanced tools related to fuzzy and neural
systems.
C5. Extend existing knowledge in designing a feedback fuzzy controller and
single/multilayer neural networks based on the given Industrial
requirements.
C7. Interpret and evaluate through written and oral forms.
C8. Lead multiple groups and projects with decision making responsibilities.
Topics
Exp 2: On McCulloch-Pitts Neuron Model
Exp 2: On McCulloch-Pitts Neuron Model
1. Aim/Objective:
What is the objective of this experiment?
2. Theory
Provide explanation on the McCulloch-Pitts Neuron Model
3. Required Components

Python programming software/ https://www.kaggle.com/

4. Procedure:
Step by step implementation
5. Program
Python program to implement the experiment
6. Result
State the output with figures
7. Conclusion
Summarize what you have learned and performed in the experiment

A very basic perceptron for classifying OR function in python

import numpy as np
Course Module
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
def step_function(x):
if x<0:
return 0
else:
return 1
training_set = [((0, 0), 0), ((0, 1), 1), ((1, 0), 1), ((1, 1), 1)]
# ploting data points using seaborn (Seaborn requires dataframe)
plt.figure(0)
x1 = [training_set[i][0][0] for i in range(4)]
x2 = [training_set[i][0][1] for i in range(4)]
y = [training_set[i][1] for i in range(4)]
df = pd.DataFrame(
{'x1': x1,
'x2': x2,
'y': y
})
sns.lmplot("x1", "x2", data=df, hue='y', fit_reg=False, markers=["o", "s"])
# parameter initialization
w = np.random.rand(2)
errors = []
eta = .5
epoch = 30
b=0
# Learning
for i in range(epoch):
for x, y in training_set:
# u = np.dot(x , w) +b
u = sum(x*w) + b
error = y - step_function(u)
errors.append(error)
for index, value in enumerate(x):
#print(w[index])
w[index] += eta * error * value
 [Introduction to Fuzzy/Neural Systems]
2
[On McCulloch-Pitts Neuron Model]

b += eta*error
''' produce all decision boundaries
a = [0,-b/w[1]]
c = [-b/w[0],0]
plt.figure(1)
plt.plot(a,c)
'''
# final decision boundary
a = [0,-b/w[1]]
c = [-b/w[0],0]
plt.plot(a,c)
# ploting errors
plt.figure(2)
plt.ylim([-1,1])
plt.plot(errors)

Program to implement McCulloch and Pitts Neuron model

# In[11]:
import numpy
import scipy.special

Course Module
import glob
import scipy.misc
class neuralNetwork:
def __init__(self, inputNodes, hiddenOneNodes, hiddenTwoNodes, hiddenThreeNodes,
finalNodes, alpha):
self.inputNodes = inputNodes
self.hiddenOneNodes = hiddenOneNodes
self.hiddenTwoNodes = hiddenTwoNodes
self.hiddenThreeNodes = hiddenThreeNodes
self.finalNodes = finalNodes
self.alpha = alpha
self.weightsInputHidden = numpy.random.normal(0.0, pow(self.hiddenOneNodes, -
0.5),(self.hiddenOneNodes,self.inputNodes))
self.weightsHiddenOneHiddenTwo = numpy.random.normal(0.0,
pow(self.hiddenTwoNodes,-0.5),(self.hiddenTwoNodes,self.hiddenOneNodes))
self.weightsHiddenTwoHiddenThree = numpy.random.normal(0.0,
pow(self.hiddenThreeNodes,-0.5),(self.hiddenThreeNodes,self.hiddenTwoNodes))
self.weightsHiddenOutput = numpy.random.normal(0.0, pow(self.hiddenOneNodes,-
0.5),(self.finalNodes, self.hiddenThreeNodes))
pass
def train(self, inputs, target):
inputs = numpy.array(inputs, ndmin=2).T
target = numpy.array(target, ndmin=2).T
hiddenInput = numpy.dot(self.weightsInputHidden,inputs)
hiddenOneOutput = self.sigmoid(hiddenInput)
hiddenTwoInput = numpy.dot(self.weightsHiddenOneHiddenTwo,hiddenOneOutput)
hiddenTwoOutput = self.sigmoid(hiddenTwoInput)
hiddenThreeInput =
numpy.dot(self.weightsHiddenTwoHiddenThree,hiddenTwoOutput)
hiddenThreeOutput = self.sigmoid(hiddenThreeInput)
finalInput = numpy.dot(self.weightsHiddenOutput,hiddenThreeOutput)
finalOutput = self.sigmoid(finalInput)
outputError = target - finalOutput
hiddenOutputError = numpy.dot(self.weightsHiddenOutput.T, outputError)
hiddenThreeHiddenTwoError = numpy.dot(self.weightsHiddenTwoHiddenThree.T,
hiddenOutputError)
hiddenTwoHiddenOneError = numpy.dot(self.weightsHiddenOneHiddenTwo.T,
hiddenThreeHiddenTwoError)
 [Introduction to Fuzzy/Neural Systems]
2
[On McCulloch-Pitts Neuron Model]

hiddenInputError = numpy.dot(self.weightsInputHidden.T,
hiddenTwoHiddenOneError)
self.weightsHiddenOutput += self.alpha * numpy.dot((outputError * finalOutput * (1.0 -
finalOutput)),numpy.transpose(hiddenThreeOutput))
self.weightsHiddenTwoHiddenThree += self.alpha * numpy.dot((hiddenOutputError *
hiddenThreeOutput * (1.0 -
hiddenThreeOutput)),numpy.transpose(hiddenTwoOutput))
self.weightsHiddenOneHiddenTwo += self.alpha *
numpy.dot((hiddenThreeHiddenTwoError * hiddenTwoOutput * (1.0 -
hiddenTwoOutput)),numpy.transpose(hiddenOneOutput))
self.weightsInputHidden += self.alpha * numpy.dot((hiddenTwoHiddenOneError *
hiddenOneOutput * (1.0 - hiddenOneOutput)),numpy.transpose(inputs))
pass
def query(self, inputs):
inputs = numpy.array(inputs, ndmin=2).T
hiddenInput = numpy.dot(self.weightsInputHidden,inputs)
hiddenOneOutput = self.sigmoid(hiddenInput)
hiddenTwoInput = numpy.dot(self.weightsHiddenOneHiddenTwo,hiddenOneOutput)
hiddenTwoOutput = self.sigmoid(hiddenTwoInput)
hiddenThreeInput =
numpy.dot(self.weightsHiddenTwoHiddenThree,hiddenTwoOutput)
hiddenThreeOutput = self.sigmoid(hiddenThreeInput)
finalInput = numpy.dot(self.weightsHiddenOutput,hiddenThreeOutput)
finalOutput = self.sigmoid(finalInput)
return finalOutput
pass
def sigmoid(self, x):
return scipy.special.expit(x)
pass
# In[12]:
#AND
n = neuralNetwork(2,12,36,12,1,0.1)
print('Before training')
print(n.query([0,0]))
print(n.query([0,1]))
Course Module
print(n.query([1,0]))
print(n.query([1,1]))
print("Training...")
for i in range(0, 10000):
n.train([0,0],[0])
n.train([0,1],[0])
n.train([1,0],[0])
n.train([1,1],[1])
print("Done")
print(n.query([0,0]))
print(n.query([0,1]))
print(n.query([1,0]))
print(n.query([1,1]))

# In[13]:
#OR
n = neuralNetwork(2,12,36,12,1,0.1)
print('Before training')
print(n.query([0,0]))
print(n.query([0,1]))
print(n.query([1,0]))
print(n.query([1,1]))
print("Training...")
for i in range(0, 10000):
n.train([0,0],[0])
n.train([0,1],[1])
n.train([1,0],[1])
n.train([1,1],[1])
print("Done")
print(n.query([0,0]))
print(n.query([0,1]))
print(n.query([1,0]))
print(n.query([1,1]))
# In[14]:
#NOT
n = neuralNetwork(1,12,36,12,1,0.1)
 [Introduction to Fuzzy/Neural Systems]
2
[On McCulloch-Pitts Neuron Model]

print('Before training')
print(n.query([0]))
print(n.query([1]))
print("Training...")
for i in range(0, 10000):
n.train([0],[1])
n.train([1],[0])
print("Done")
print(n.query([0]))
print(n.query([1]))
# In[15]:
#NAND
n = neuralNetwork(2,12,36,12,1,0.1)
print('Before training')
print(n.query([0,0]))
print(n.query([0,1]))
print(n.query([1,0]))
print(n.query([1,1]))
print("Training...")
for i in range(0, 10000):
n.train([0,0],[1])
n.train([0,1],[1])
n.train([1,0],[1])
n.train([1,1],[0])
print("Done")
print(n.query([0,0]))
print(n.query([0,1]))
print(n.query([1,0]))
print(n.query([1,1]))
# In[16]:
#XOR
n = neuralNetwork(2,12,36,12,1,0.1)

Course Module
print('Before training')
print(n.query([0,0]))
print(n.query([0,1]))
print(n.query([1,0]))
print(n.query([1,1]))
print("Training...")
for i in range(0, 10000):
n.train([0,0],[0])
n.train([0,1],[1])
n.train([1,0],[1])
n.train([1,1],[0])
print("Done")
print(n.query([0,0]))
print(n.query([0,1]))
print(n.query([1,0]))
print(n.query([1,1]))
<Exercise 1.
Lab Activity: Simulation
Design and develop the neural network system for the following experiment
Experiment 1: McCulloch and Pitts Network
1. Design and train a neural network system which can perform AND and OR
operation.
2. Tune the neural network model and minimize the error by updating the
weights and perform the testing.
3. Run the simulation in group and explain the working principles of the
algorithm.
4. Interpret the output of the designed neural network system by varying the
inputs.

References and Supplementary Materials

Books and Journals

1. Van Rossum, G. (2007). Python Programming Language. In USENIX annual technical

conference (Vol. 41, p. 36).
2. SN, S. (2003). Introduction to artificial neural networks.
3. Rashid, T. (2016). Make your own neural network. CreateSpace Independent Publishing
Platform.

Online Supplementary Reading Materials

 [Introduction to Fuzzy/Neural Systems]
2
[On McCulloch-Pitts Neuron Model]

1. Chaitanya Singh; How to Install Python. (n.d.). Retrieved 14 May 2020, from
https://beginnersbook.com/2018/01/python-installation/; 14-05-2020

Course Module