Scribd
Upload
55 views

Machine Learning Notes

1. The sigmoid activation function is commonly used in neural networks because the relationship between its value and derivative reduces computational burden during training. 2. There are two types of sigmoid functions: the logistic sigmoid which ranges from 0 to 1, and the bipolar sigmoid which ranges from -1 to 1. 3. A perceptron is a basic unit of an artificial neural network that takes inputs, calculates a linear combination, and outputs 1 if the result is above a threshold and -1 otherwise. It is trained using the perceptron training rule to update weights based on errors.

Uploaded by

Yash Gandharv
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
55 views

Machine Learning Notes

1. The sigmoid activation function is commonly used in neural networks because the relationship between its value and derivative reduces computational burden during training. 2. There are two types of sigmoid functions: the logistic sigmoid which ranges from 0 to 1, and the bipolar sigmoid which ranges from -1 to 1. 3. A perceptron is a basic unit of an artificial neural network that takes inputs, calculates a linear combination, and outputs 1 if the result is above a threshold and -1 otherwise. It is trained using the perceptron training rule to update weights based on errors.

Uploaded by

Yash Gandharv
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 5

Sigmodal function:

It is widely used in back propagation network because of the relationship between the value of the
function at a point and the value of the derivative at that point which reduces the computational
burden during training.

It is divided into 2 types: -

1. Binary sigmoid or logistic sigmoid or Uni-polar sigmoid:

Sigmoid function is defined as

f(x) = 1/1+e^-λx

where λ is stiffness parameter

Range of sigmoid function varies from 0 to 1

2. Bi-polar sigmoid: It is defined as

f(x) = 2/1+e^λx - 1

1-e^-λx/1+e^λx

f'(x) = λ/2[1+f(x)][1-f(x)]
Range of sigmoid function varies from -1 to 1

a) Sigmoid activation function:

We can obtain the output of the neuron, for binary sigmoidal function and bi-polar sigmoidal
activation function

b) Bi-polar sigmoidal activation function:


Perceptron in Artificial Neural Network:

A perceptron is used to build the artificial neural network system. It takes a vector of real-valued
inputs and calculates linear combination of these inputs then output is 1 if the result is greater than
some threshold value else -1 otherwise.

Inputs: x1 to xn and the output ~ O(x1 …. xn)

Computed by the perception is

O(x1, x2,….,xn) = 1 if w0 + x1w1 + x2w2 + …. + xnwn > 0

-1 if otherwise

Updation of weights is called Perceptron Training Rules

If Output is equal to target O/P then don’t update weights

If O/P is not equal to target O/P then update weights

Perceptron Training Rules:

1. One way to learn an acceptable weight vector is to begin with random weights, then
iteratively apply the perceptron to each training example, modify the perceptron weights
whenever it misclassifies an example.
2. This process is repeated iterating through the training examples as many times as needed
until the perceptron classifies all training examples correctly.
3. Weights are modified at each step according to the perceptron training rule, which revises
the weights ‘wi’ associated with input ‘xi’ according to the rule

Wi <- wi + ∆ wi

Where ∆ wi = n( t – o ) xi

n -> learning rate = 0.1

t = target O/P

o = Actual O/P

xi = Input
Algorithm:

perceptron_training_rule(X,n)

Initialise w(small random values)

repeat

for each training instance(x,tx) ∑ X

Compute the real OX = Actual O/P

∑ (w,x)

if (tx ≠ ox)

for each wi

wi <- wi + ∆wi

∆ wi < n(tx-ox) xi
end for

end if

end for

Note: Until all training instances in X are correctly classified then you have to return w

Implementation of logical AND gate using perceptron end rule:

Truth Table:

x1 x2 x1 ^ x2
0 0 0
0 1 0
1 0 0
1 1 1

w1 = 1.2, w2 = 0.6, VT = 1

Learning rate n = 0.5

x1 = 0, x2 = 0, tx = 0 (If wixi ≥ 1 then o/p is 1, if wixi ≤ 1 then o/p is 0)

1. wixi = w1x1 + w2x2 = (1.2 x 0) + (0.6 x 0) = 0 ≥ 1 => tx = 0

ox = 0 (see fig) tx = 0 (see fig) => tx = ox

2. wixi = w1x1 + w2x2 = (1.2 x 0) + (0.6 x 1) = 0.6 ≥ 1 => tx = 0

ox = 0 (see fig) tx = 0 (see fig) => tx = ox

3. wixi = w1x1 + w2x2 = (1.2 x 1) + (0.6 x 0) = 1.2 ≥ 1 => 1

ox = 1 (see fig) tx = 0 (see fig) => tx ≠ ox


weights should be updated

wi = w1 + n(tx-ox)xi

w1 = 1.2 + 0.5(0-1)1 = 0.7

w2 = 0.6 + 0.5(0-1)0 = 0.6

w1 = 0.7, w2 = 0.6

i) x1 = 0 & x2 = 0, tx = 0
wixi = w1x1 + w2x2 = 0.7 x 0 + 0.6 x 0 = 0
tx = ox
ii) x 1 = 0 & x2 = 1
tx = ox
iii) x1 = 1, x2 = 1, tx = 1
tx = ox
iv) x1 = 1, x2 = 1, tx = 1
wixi = w1x1 + w2x2 = 0.7 x 1 + 0.6 x 1 = 1.3 ≥ 1
tx = 1 => tx = ∆x

You might also like