Perceptron learning

Presentation outline

Historical Background
 Perceptron Architecture
Constructing perceptron Learning Rules
Literature on perceptron learning algorithm
 Historical back ground
Biological Inspiration

The perceptron, a fundamental unit in artificial neural networks, draws

inspiration from the biological workings of neurons in the brain.

Neurons receive input signals from other neurons through dendrites.

These signals have varying strengths, influencing the neuron's overall activation
level.
Relation between ANN and BNN
 Historical background
The perceptron was first introduced by McCulloch and Pitts in 1943 as a
mathematical model of biological neurons.

In 1957, Frank Rosenblatt built the first hardware implementation of a perceptron

In the late 1960s, Marvin Minsky and Seymour Papert showed that perceptron
could not learn many important classes of functions.

Multi layer perceptron, a type of machine learning that uses deep neural
networks, wide use, including image recognition, speech recognition, and
natural language processing
 Historical background cont..

The perceptron algorithm was initially used to solve simple problems, such as recognizing
handwritten characters, but it soon faced criticism due to its limited capacity to learn
complex patterns and cannot handle non-linearly separable data.

Due to this, decline research on perceptron in the 1960s and 1970s.

In the 1980s, the development of backpropagation, a powerful algorithm for training multi-
layer neural networks developed, renewed interest in ANN and innovation in machine
learning.
 Perceptron Model in Machine Learning
It has four main parameters: input values, weights and Bias, net sum, and an activation
function .
Types of Perceptron models
a) Single Layer Perceptron model:
 One of the easiest ANN types consists of a feed-forward network and includes a threshold
transfer inside the model.
 The main objective of the single-layer perceptron model is to analyze the linearly separable
objects with binary outcomes.
 can learn only linearly separable patterns.
b) Multi-Layered Perceptron model:
 It is mainly similar to a single-layer perceptron model but has more hidden layers.
i. Forward stages:
ii. Backward stages
 Perceptron Architecture
 Architecture is a pattern of connections between neurons

Architecture

Single layer single neurons

perceptron
& Multi-Layer multi neurons
(feedforward)
Single layer multi neurons perceptron
Single layer single neuron
 Perceptron Architecture
Multi-Layer - multi input feedforward Perceptron
 Neural networks Learning Rules

Learning rule is a procedure for modifying the weight and biases of a network.

The rules can be classified into three broad categories:

Supervised learning
Unsupervised learning
Reinforcement learning
 Machine learning algorithm
Supervised learning Unsupervised learning Reinforcement learning
A machine is train using unlabeled An agent is interacting with the
Teaching a machine using data environment by producing action
label data and discover s error and rewards
No predefined data
Re • Predicting a continuous
• Finding pattern and data
to find occurrences b/n
gr quantity Asso them
• Training models through
• like items that are occurs
es • Number of possibility and ciati
on together trial and error to learn
si predict a single output
. optimal behavior in an
on
Cla environment.
• Predicting a label or class Clus • Division of objects
ssif teri
icat ng • Algorithm:
ion • Linear regression/ • Algorithm :
Al
• Logistic regression . • Q-learning.
• Support vector machine • K-means
go .
rit • K-nearest neighbor • C-means
• Random forest
h (e.g. in control and medical system
m
(e.g., training robots for navigation) 11
e.g., image recognition)
Overview perceptron Learning
Uses supervised learning to adjust its weights in response to a comparative signal between the
network’s actual output and the target output.
Mainly designed to classify linearly separable patterns

Patterns are linearly separable means that there exists a hyper-planar multidimensional decision
boundary that classifies the patterns into two classes.

Linearly Separable Patterns Non-Linearly Separable Patterns

 Constructing perceptron Learning Rules

Where: X=input vector, W=synaptic weights

n = WX + b
, b= threshold(biases), a= actual output,
a = f (WX + b) f = activation (transfer) function
Constructing perceptron Learning Rules
 Constructing perceptron Learning Rules, cont.…

, b=, X=

Perceptron training begins by assigning some initial values for the networks parameters.
To update the ith row of the weight matrix use:
wnew = wold + e p
To update the ith element of the bias vector use:
b new = b old + e
e = t–a
 General overview of training algorithm
.

 To update the ith row of the weight matrix use:

wnew = wold + e p

To update the ith element of the biases vector

b new = b old + e
 Perceptron Learning Algorithm
1. Initialized weights and bias (thresholds) to random values

2. Choose an input-output pattern (x,t) from the training data

3. Compute the actual output (a)

4. Compute the error (e=t-a)

5. Adjust the weights and bias according to the perceptron learning rule

6. If a whole epoch is complete, the pass to the following step, otherwise go to step 2

7. If the weight or and bias reached steady state( stop learning otherwise go to step 2
 Perceptron learning mechanism
Some of the training rule algorithm are:
a) Gradient of descent :

finds the minimum of an objective function by incrementally updating its

parameters in the negative direction of the derivative of this function
It goals is to minimize the cost function
Mathematically ;
=, E= (
b) Delta rule:
By derivative of the cost function
=
Researchers
Literature onresearch Title
perceptron Methods
learning they used
algorithm

Jayshril s. Prediction of Data source: database consists of 303 records with each having 13 clinical
Sonawane; Heart Disease attributes that include age, sex, type of chest pain, resting blood pressure,
D. R. Patil Using Multilayer cholesterol, blood sugar, resting, ECG, maximum heart rate, exercise induced
Perceptron angina, old peak, number of vessels coloured
Neural Network Learning consists of two steps,
 First step 13 clinical attributes are
accepted as input
 Three layers and feed forward neural network model
 Used multi-layer perceptron learning algorithm by back-propagation learning
algorithm
 Findings : number of neurons increase the error reduced
 When the neurons is 5, the error 7.07, while the numbers of neuron increase
to 20 the error reduced to 1.41