Perceptron Notes

Download as docx, pdf, or txt
Download as docx, pdf, or txt
You are on page 1of 4

What is Perceptron?

A perceptron is the smallest element of a neural network. Perceptron is a single-layer neural network
linear or a Machine Learning algorithm used for supervised learning of various binary classifiers. It
works as an artificial neuron to perform computations by learning elements and processing them for
detecting the business intelligence and capabilities of the input data. A perceptron network is a group
of simple logical statements that come together to create an array of complex logical statements,
known as the neural network.

Components of a Perceptron

Each perceptron comprises four different parts:


1. Input Values: A set of values or a dataset for predicting the output value. They are also
described as a dataset’s features and dataset.
2. Weights: The real value of each feature is known as weight. It tells the importance of that
feature in predicting the final value.
3. Bias: The activation function is shifted towards the left or right using bias. You may
understand it simply as the y-intercept in the line equation.
4. Summation Function: The summation function binds the weights and inputs together. It is a
function to find their sum.
5. Activation Function: It introduces non-linearity in the perceptron model.

Why do we Need Weight and Bias?


Weight and bias are two important aspects of the perceptron model. These are learnable parameters
and as the network gets trained it adjusts both parameters to achieve the desired values and the
correct output.

Weights are used to measure the importance of each feature in predicting output value. Features with
values close to zero are said to have lesser weight or significance. These have less importance in the
prediction process compared to the features with values further from zero known as weights with a
larger value. Besides high-weighted features having greater predictive power than low-weighting
ones, the weight can also be positive or negative. If the weight of a feature is positive then it has a
direct relation with the target value, and if it is negative then it has an inverse relationship with the
target value.
Perceptron Learning Rule
The late 1950s saw the development of a new type of neural network called perceptrons, which were
similar to the neurons from an earlier work by McCulloch and Pitts. One key contribution by Frank
Rosenblatt was his work for training these networks with perceptron learning rules. According to the
rule, perceptron can learn automatically to generate the desired results through optimal weight
coefficients.

Rosenblatt defined four perceptron learning rules that can be classified as follows:
Supervised Learning Algorithms
1. Gradient Descent- In order to optimize the weights of a perceptron in a machine learning
model, there needs to be an adjustable function that can predict future outcomes. Weights and
activation functions help with error reduction. Activation functions come into play because
they help determine how much weight should go towards each input when prediction errors
are calculated. The more differentiable it becomes at predicting values based on past statistics
about samples within its domain (trained data) the better it will be able to estimate accurate
answers.

In this learning, the error gradient E impacts the adjustment of weights. An example of this learning
is the backpropagation rule.

2. Stochastic- The term Stochastic is a mathematical term that refers to a variable process or an
outcome that involves randomness and uncertainty. The perceptron in machine learning
adjusts weights in a probabilistic fashion under this rule.
Unsupervised Learning Algorithms
1. Hebbian- A perceptron learning rule was proposed by Hebb in 1949. It uses a weight matrix,
W to perform correlative adjustment of weights. Weight adjustment is done by transposing
the output.

2. Competitive- In the perceptron learning algorithm, when an input pattern is sent to the entire
layer of neurons all the units compete for space and time. The only way that a neuron can win
against others in this type of competition is by having more efficient weights.

Single Layer Perceptron Model


A single-layer perceptron model is the simplest type of artificial neural network. It includes a feed-
forward network that can analyze only linearly separable objects while being dependent on a
threshold transfer function. The model returns only binary outcomes(target) i.e. 1, and 0.
The algorithm in a single-layered perceptron model does not have any previous information initially.
The weights are allocated inconsistently, so the algorithm simply adds up all the weighted inputs. If
the value of the sum is more than the threshold or a predetermined value then the output is delivered
as 1 and the single-layer perceptron is considered to be activated.

When the values of input are similar to those desired for its predicted output, then we can say that the
perceptron has performed satisfactorily. If there is any difference between what was expected and
obtained, then the weights will need adjusting to limit how much these errors affect future predictions
based on unchanged parameters.

However, since the single-layer perceptron is a linear classifier and it does not classify cases if they
are not linearly separable. So, due to the inability of the perceptron to solve problems with linearly
non-separable cases, the learning process will never reach the point with all cases properly classified.

Multilayer Perceptron Model

A multi-layer perceptron model uses the backpropagation algorithm. Though it has the same structure
as that of a single-layer perceptron, it has one or more hidden layers.
The backpropagation algorithm is executed in two phases:
 Forward phase- Activation functions propagate from the input layer to the output layer. All
weighted inputs are added to compute outputs using the sigmoid threshold.
 Backward phase- The errors between the observed actual value and the demanded nominal
value in the output layer are propagated backward. The weights and bias values are modified to
achieve the requested value. The modification is done by apportioning the weights and bias to
each unit according to its impact on the error.

Perceptron Function
Perceptron function ”f(x)” is generated by multiplying the input ‘x’ with the learned weight
coefficient ‘w’. The same can be expressed through the following mathematical equation:

You might also like