Module-2

Dr. Bahubali Shiragaur

Director @ Schhol of Computer Science, Engg. & Applications
D Y Patil International University Akrudi Pune
 Can anyone identify this animal ?
Exactly …. It’s dog..

Why not CAT ? Or Fox? Or Lion ?

Brain is trained…

What kind of system is present inside the Brain?

YES… Nervous System

Contents
• Introduction ANN
• Neural Network
• Artificial Neural
Network
• Why ANN ?
• History ANN
• Basics of NN
• Learning Rules
• Hands on google Collab
 How This Human Nervous System Works?
Inpu Outpu
Processing
t t

1. Dendrites = Input
2. Nucleus / Soma / Neuron =
processes the information
3. Axon = Output
4. Axon terminals / Synapses = Point
of connection to other neurons
McCULLOCH PITTS NEURON MODEL
• Mankind’s First Mathematical
Model of A Biological Neuron
• Its binary activated
• Output is only 0 or 1

• Neurons are connected by

Directed weighted paths
• It has 2 types of paths
• Excitatory (Positive weights)
• Inhibitory (negative weights)

• Neuron is associated with a

threshold value
• Neuron fires only if + + + +
net input > threshold
• Threshold is set so that +
inhibitory is absolute.
• Non zero inhibitory input prevents
the neuron from firing Y=f(y =
Starter
• Can Machines learn ?
• Can I teach my machines to mimic AND gate
 AND Function using McCulloch Pitts Neuron
Model
Y=f(Z
• Net input or weighted sum is given by X1 1
Y
Z= Z
Z=
Z=1*+1* X2 1

Z= +
Y=f(Z = X1 X2 Y
0 0 0
0 1 0
1 0 0
1 1 1
AND Function using Bipolar inputs

X1 1
Y
x2 Z

x1 X2 1

x2 = 1 – x1
X1 X2 Y
1 1 1

x1 + x2 = z 1 -1 -1
As z = 1 -1 1 -1
x1 + x2 = 1 -1 -1 -1
x2 = 1 – x1
Linearly Separable !!!
Can ..
• The solution be unique ?
• There be a guarantee that solution exits ?
• There be a guarantee that the solution is optimum solution ?
X-OR Function using Bipolar inputs

x2
X1 3
Y
x1 Z

X2 3

X1 X2 Y
1 1 -1
Z = 3x1 + 3x2 1 -1 1
Y=f(Z = 3x1-3x2=1---1
Non Linearly Separable !!! -3x1+3x2=1--2 -1 1 1
-1 -1 -1
 What is a Perceptron?
A perceptron is a binary classification algorithm modeled after the functioning of the human
brain
What is Multilayer Perceptron?
• A multilayer perceptron (MLP) is a group of perceptrons, organized in multiple layers, that can
accurately answer complex questions.
• Each perceptron in the first layer (on the left) sends signals to all the perceptrons in the second
layer, and so on.
• An MLP contains an input layer, at least one hidden layer, and an output layer.
 The Perceptron Learning Process
• Takes the inputs which are fed into the
perceptrons in the input layer, multiplies them
by their weights, and computes the sum.
Z=
• Adds the number one, multiplied by a “bias
weight”.
Z = + b*1
• This is a technical step that helps the output of
each perceptron to move up, down, left and
right based on this value the activation
function makes a decision. b
• Feeds the sum through the activation function
in a simple perceptron system, the activation
function is a step function.
• The result of the step function is the output. 1
Y=f(Z =
 Perceptron Learning Algorithm
• S1 : Initialize weights & Bias (Initially set to 0)
• S2 : Compute the response of the net
X1 w1
• S3 : Apply Activation function
Y
• S4: check for stopping condition : Z
• if false update weight & Bias
• S5 : Repeat S2-S4
X2 w2
Z= b + b
Y=f(Z =
Y = t : where t = target 1 Activation of this
• + unit is always 1

where = *t*
• + Perceptron is the basic unit of the neural network.
where = t*
Accepts an input and generates a prediction.

• Assume
Artificial Neural Network (ANN)
 ANN is an information processing system.
 Elements called Neurons process the information : Heart of ANN
 Neural Net can be single layer or multi-layer (MNN) net
 ANN thinks and behaves exactly like human.
 It has 2 passes

1. Forward pass
2. Backward pass
 Backward pass updates weight
 Artificial Neural Networks (ANN)
&
Deep Neural Network (DNN)
• Artificial Neural Networks (ANN) is a supervised learning system built of a large number of
simple elements, called neurons or perceptron's.
• Each neuron makes a simple decisions, and feeds those decisions to other neurons, organized
in interconnected layers.
• Together, the neural network can implement almost any function, and practically answer any
questions, given enough training samples and computing power.
• A “shallow” neural network has only three layers of neurons:
1. An input layer that accepts the independent variables or inputs of the model
2. One hidden layer that connects the input & output layers
3. An output layer that generates predictions
• A Deep Neural Network (DNN) has a similar structure, but it has two or more “hidden layers” of
neurons that process inputs.
• As more number of neuron layers are added, deep learning networks are more accurate, and
also improve in accuracy.
Comparison
Difference between MLP and ANN

• In MLP the decision/ activation function is a step function and the

output is binary.
• Artificial Neural Networks are evolved from MLPs.
• Other activation functions can be used in ANN which result in outputs
of real values, usually between 0 and 1 or between -1 and 1.
• ANN allows for probability-based predictions (softmax) or classification
of items into multiple labels (tanH).
Module-2 CNN Architecture
 Fashion Class Classification Using Deep
Learning
•# STEP #0: PROBLEM STATEMENT
•Fashion training set consists of 70,000 images divided into 60,000 training and 10,000
testing samples. Dataset sample consists of 28x28 grayscale image, associated with a label
from 10 classes.
•The 10 classes are as follows:
• 0 => T-shirt/top
• 1 => Trouser
• 2 => Pullover
• 3 => Dress
• 4 => Coat
• 5 => Sandal
• 6 => Shirt
• 7 => Sneaker
• 8 => Bag
• 9 => Ankle boot

•Each image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total.
Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of
that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and
255.
Continue…
Predict Fashion Classes

Click icon to add picture

Cont..
What are CNN?
Feature Detector
Setosa live example
• Live Convolution: http://setosa.io/ev/image-kernels/
Feature Detectors
ReLu – Rectified Linear Units

Click icon to add picture

Relu
POOLING (DOWNSAMPLING)

Click icon to add picture

http://scs.ryerson.ca/~aharley/vis/conv/flat.html
 How to improve model?
Increase filters /Drop outs
Continue
STEP #1: IMPORT LIBRARIES
STEP #2: LOAD DATASET
STEP #3: DATA VISUALIZATION
STEP#4: BUILD AND TRAIN A DEEP LEARNING MODEL
STEP #5: EVALUATE THE MODEL
THANK YOU