0% found this document useful (0 votes)
71 views18 pages

Adaline and Delta Learning Rule

The document applies the Adaline learning algorithm to train a neural network to perform different logic functions on binary and bipolar inputs. It traces the algorithm over one epoch with a learning rate of 0.1 and initial weights of 0.1. For each logic function, it shows the input, target, calculation of network output, error, and update of weights and bias. The functions tested include OR, AND, and their combinations with binary and bipolar inputs/outputs. The network output and error decrease over the iterations, showing the network is learning the function from the training examples.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
71 views18 pages

Adaline and Delta Learning Rule

The document applies the Adaline learning algorithm to train a neural network to perform different logic functions on binary and bipolar inputs. It traces the algorithm over one epoch with a learning rate of 0.1 and initial weights of 0.1. For each logic function, it shows the input, target, calculation of network output, error, and update of weights and bias. The functions tested include OR, AND, and their combinations with binary and bipolar inputs/outputs. The network output and error decrease over the iterations, showing the network is learning the function from the training examples.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

Q2: Apply Adaline learning algorithm for the OR function of bipolar input

and bipolar output with learning rate (α=0.1) initial weights=0.1. Trace with
one epoch.

INPUT TARGET
1 1 1
1 -1 1
-1 1 1
-1 -1 -1

𝑦𝑖𝑛 = 𝑏 + ∑ 𝑊𝑖 𝑋𝑖

𝑊𝑛𝑒𝑤 = 𝑊𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )𝑋

𝑏𝑛𝑒𝑤 = 𝑏𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )


X1 X2 b t 𝑦𝑖𝑛 t-𝑦𝑖𝑛 ∆𝑊1 ∆𝑊2 ∆𝑏 𝑊1 𝑊2 b ERROR
1 1 1 1 0.3 0.7 0.07 0.07 0.07 0.17 0.17 0.17 0.49
1 -1 1 1 0.17 0.83 0.083 -0.083 0.083 0.253 0.087 0.253 0.6889
-1 1 1 1 0.087 0.913 -0.0913 0.0913 0.0913 0.1617 0.1783 0.3443 0.83356
-1 -1 1 -1 0.0043 -1.0043 0.10043 0.10043 -0.10043 0.26213 0.2787 0.2439 1.0086
Q2: Apply Adaline learning algorithm for the OR function of binary input
and bipolar output with learning rate (α=0.1) initial weights=0.1. Trace with
one epoch.

INPUT TARGET
1 1 1
1 0 1
0 1 1
0 0 -1

𝑦𝑖𝑛 = 𝑏 + ∑ 𝑊𝑖 𝑋𝑖

𝑊𝑛𝑒𝑤 = 𝑊𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )𝑋

𝑏𝑛𝑒𝑤 = 𝑏𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )


X1 X2 b t 𝑦𝑖𝑛 t-𝑦𝑖𝑛 ∆𝑊1 ∆𝑊2 ∆𝑏 𝑊1 𝑊2 b ERROR
1 1 1 1 0.3 0.7 0.07 0.07 0.07 0.17 0.17 0.17 0.49
1 0 1 1 0.34 0.66 0.066 0 0.066 0.236 0.17 0.236 0.4356
0 1 1 1 0.406 0.594 0 0.0594 0.0594 0.236 0.2294 0.2954 0.352836
0 0 1 -1 0.2954 0.7046 0 0 0.07046 0.236 0. 2294 0.3658 0.4964
Q2: Apply Adaline learning algorithm for the OR function of binary input
and binary output with learning rate (α=0.1) initial weights=0.1. Trace with
one epoch.

INPUT TARGET
1 1 1
1 0 1
0 1 1
0 0 0

𝑦𝑖𝑛 = 𝑏 + ∑ 𝑊𝑖 𝑋𝑖

𝑊𝑛𝑒𝑤 = 𝑊𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )𝑋

𝑏𝑛𝑒𝑤 = 𝑏𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )


X1 X2 b t 𝑦𝑖𝑛 t-𝑦𝑖𝑛 ∆𝑊1 ∆𝑊2 ∆𝑏 𝑊1 𝑊2 b ERROR
1 1 1 1 0.3 0.7 0.07 0.07 0.07 0.17 0.17 0.17 0.49
1 0 1 1 0.34 0.66 0.066 0 0.066 0.236 0.17 0.236 0.4356
0 1 1 1 0.406 0.594 0 0.0594 0.0594 0.236 0.2294 0.2954 0.352836
0 0 1 0 0.2954 -0.2954 0 0 -0.02954 0.236 0. 2294 0.2658 0.087
Q2: Apply Adaline learning algorithm for the AND function of bipolar input
and bipolar output with learning rate (α=0.1) initial weights=0.1. Trace with
one epoch.

INPUT TARGET
1 1 1
1 -1 -1
-1 1 -1
-1 -1 -1

𝑦𝑖𝑛 = 𝑏 + ∑ 𝑊𝑖 𝑋𝑖

𝑊𝑛𝑒𝑤 = 𝑊𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )𝑋

𝑏𝑛𝑒𝑤 = 𝑏𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )


X1 X2 b t 𝑦𝑖𝑛 t-𝑦𝑖𝑛 ∆𝑊1 ∆𝑊2 ∆𝑏 𝑊1 𝑊2 b ERROR

1 1 1 1 0.3 0.7 0.07 0.07 0.07 0.17 0.17 0.17 0.49


1 -1 1 -1 0.17 -1.17 -0.117 0.117 -0.117 0.053 0.287 0.053 1.3689
-1 1 1 -1 0.053 -1.053 0.1053 -0.1053 0.1053 0.1583 0.1817 0.1583 1.1088
-1 -1 1 -1 -0.1817 -0.8183 0.08183 0.08183 -0.08183 0.240 0.2635 0.0764 0.6696
Q2: Apply Adaline learning algorithm for the AND function of binary input
and bipolar output with learning rate (α=0.1) initial weights=0.1. Trace with
one epoch.

INPUT TARGET
1 1 1
1 0 -1
0 1 -1
0 0 -1

𝑦𝑖𝑛 = 𝑏 + ∑ 𝑊𝑖 𝑋𝑖

𝑊𝑛𝑒𝑤 = 𝑊𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )𝑋

𝑏𝑛𝑒𝑤 = 𝑏𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )


X1 X2 b t 𝑦𝑖𝑛 t-𝑦𝑖𝑛 ∆𝑊1 ∆𝑊2 ∆𝑏 𝑊1 𝑊2 b ERROR

1 1 1 1 0.3 0.7 0.07 0.07 0.07 0.17 0.17 0.17 0.49


1 0 1 -1 0.34 -1.34 -0.134 0 -0.134 0.036 0.17 0.036 1.7956
0 1 1 -1 0.206 -1.206 0 -0.1206 -0.1206 0.036 0.0494 -0.0846 1.4544
0 0 1 -1 -0.0846 -1.0846 0 0 -0.10846 0.036 0.0494 -0.193 1.176
Q2: Apply Adaline learning algorithm for the AND function of binary input
and binary output with learning rate (α=0.1) initial weights=0.1. Trace with
one epoch.

INPUT TARGET
1 1 1
1 0 0
0 1 0
0 0 0

𝑦𝑖𝑛 = 𝑏 + ∑ 𝑊𝑖 𝑋𝑖

𝑊𝑛𝑒𝑤 = 𝑊𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )𝑋

𝑏𝑛𝑒𝑤 = 𝑏𝑜𝑙𝑑 + 𝛼(𝑡 − 𝑦𝑖𝑛 )


X1 X2 b t 𝑦𝑖𝑛 t-𝑦𝑖𝑛 ∆𝑊1 ∆𝑊2 ∆𝑏 𝑊1 𝑊2 b ERROR

1 1 1 1 0.3 0.7 0.07 0.07 0.07 0.17 0.17 0.17 0.49


1 0 1 0 0.17 -0.17 -0.017 0 -0.017 0.153 0.17 0.153 0.0289
0 1 1 0 0.323 -0.323 0 -0.0323 -0.0323 0.153 0.1377 0.1207 0.1043
0 0 1 0 0.1207 -0.1207 0 0 0.01207 0.153 0.1377 0.13277 0.01456
Q1: Train a neural network with Delta Rule algorithm using the following:
2 −1 0.5
̅̅̅ ̅̅̅ ̅̅̅̅̅
𝑥1 = [ 3 ], d1=1, 𝑥2 = [ 0 ], d2=-1, initial weights is 𝑊1 = [ 1 ].
−1 2 −0.5
C=0.15, λ=1. Use continuous bipolar activation function 𝑓 (𝑛𝑒𝑡) =
2
−1
1+𝑒 −𝜆 𝑛𝑒𝑡

𝜆
given that 𝑓 ′ = (1 − 𝑓 2) (20 marks)
2

2
𝑛𝑒𝑡1 = 𝑊1𝑇 ∗ 𝑋1 = [0.5 1 −0.5] [ 3 ]=4.5
−1
2
𝑂1 = 𝑓(𝑛𝑒𝑡1) = − 1 = 0.978, 𝑑𝑖 ≠ 𝑂𝑖
1 − 𝑒 −4.5
Therefore we obtain updated weight vector
1 1
𝑓 ′ (𝑛𝑒𝑡1) = 2 (1 − (𝑂1)2 = 2 (1 − (0.978)2 ) =0.02173246

Δ𝑤1 = 𝐶 ∗ (𝑑1 − 𝑂1) ∗ 𝑓 ′ (𝑛𝑒𝑡1)*X1

2 0.000143
Δ𝑤1 = 0.15 ∗ (1 − 0.978) ∗ 0.02173 ∗ [ 3 ] = [ 0.000215 ]
−1 −0.000072
0.5 0.000143 0.500143
𝑊2 = 𝑊1 + Δ𝑤1 = [ 1 ] + [0.000215] = [ 1.000215 ]
−0.5 0.000072 −0.500072

Step2: Second sample X2


−1
𝑇
𝑛𝑒𝑡2 = 𝑊2 ∗ 𝑋2 = [0.500143 1.000215 −0.500072] [ 0 ]= -1.50029
2
2
𝑂2 = 𝑓(𝑛𝑒𝑡2) = − 1 = −0.635, 𝑑𝑖 ≠ 𝑂𝑖
1 − 𝑒 −(−1.50029)
Therefore we obtain updated weight vector
1 1
𝑓 ′ (𝑛𝑒𝑡2) = 2 (1 − (𝑂2)2 ) = 2 (1 − (−0.635)2 ) =0.29823862

Δ𝑤2 = 𝐶 ∗ (𝑑2 − 𝑂2) ∗ 𝑓 ′ (𝑛𝑒𝑡2)*X2

−1 0.016318
Δ𝑤2 = 0.15 ∗ (−1 − (−0.635)) ∗ 0.298238 ∗ [ 0 ] = [ 0 ]
2 −0.03264
0.500143 0.016318 0.516451
𝑊3 = 𝑊2 + Δ𝑤2 = [ 1.000215 ] + [ 0 ] = [ 1.000215 ]
−0.500072 −0.03264 −0.53271

___________________________________________________________

Q2: Train a neural network with Delta Rule algorithm using the following:

3 1 0.5
̅̅̅ ̅̅̅ ̅̅̅̅̅
𝑥1 = [ 2 ], d1=1, 𝑥2 = [0], d2=-1, initial weights is 𝑊1 = [ 1 ].
−1 3 −1
1
C=0.1 , λ=1. Use sigmoid activation function 𝑓 (𝑛𝑒𝑡) =
1+𝑒 −𝜆 𝑛𝑒𝑡

Given that 𝑓 ′ = 𝜆(𝑓 − 𝑓 2 ) (15 marks)


3
𝑛𝑒𝑡1 = 𝑊1𝑇 ∗ 𝑋1 = [0.5 1 −1] [ 2 ]=4.5
−1
1
𝑂1 = 𝑓(𝑛𝑒𝑡1) = = 0.989, 𝑑𝑖 ≠ 𝑂𝑖
1 − 𝑒 −4.5
Therefore we obtain updated weight vector
1
𝑓 ′ (𝑛𝑒𝑡1) = (𝑂1 − (𝑂1)2 = 2 (0.989 − (0.989)2 ) =0.01086623

Δ𝑤1 = 𝐶 ∗ (𝑑1 − 𝑂1) ∗ 𝑓 ′ (𝑛𝑒𝑡1)*X1

3 0.000036
( )
Δ𝑤1 = 0.1 ∗ 1 − 0.989 ∗ 0.01086623 ∗ [ 2 ] = [ 0.000024 ]
−1 −0.000012
0.5 0.000036 0.500036
𝑊2 = 𝑊1 + Δ𝑤1 = [ 1 ] + [ 0.000024 ] = [ 1.000024 ]
−1 −0.000012 −1.000012

Step2: Second sample X2

1
𝑛𝑒𝑡2 = 𝑊2𝑇 ∗ 𝑋2 = [0.500036 1.000024 −1.000012] [0]= -2.5
3
1
𝑂2 = 𝑓 (𝑛𝑒𝑡2) = = 0.076, 𝑑𝑖 ≠ 𝑂𝑖
1 − 𝑒 −(−2.5)
Therefore we obtain updated weight vector

𝑓 ′ (𝑛𝑒𝑡2) = (𝑂 − (𝑂2)2 ) = (0.076 − (0.076)2 ) = 0.07010372


Δ𝑤2 = 𝐶 ∗ (𝑑2 − 𝑂2) ∗ 𝑓 ′ (𝑛𝑒𝑡2)*X2

1 −0.0075
Δ𝑤2 = 0.1 ∗ (−1 − (0.076)) ∗ (0.07010372) ∗ [0] = [ 0 ]
3 −0.0226
0.500036 −0.0075 0.4924
𝑊3 = 𝑊2 + Δ𝑤2 = [ 1.000024 ] + [ 0 ] = [ 1.000024 ]
−1.000012 −0.0226 −1.02264

_____________________________________________________________

Q3: Train a neural network with Delta Rule algorithm using the following:

3 −1 0.5
̅̅̅ = [ 2 ], 𝑥2
𝑥1 ̅̅̅ = [ 0 ], initial weights is ̅̅̅̅̅
𝑊1 = [ 1 ].
−1 2 −1
C=0.15, λ=1, d1= 1, d2=-1 Use sigmoid activation function 𝑓 (𝑛𝑒𝑡) =
1
1+𝑒 −𝜆 𝑛𝑒𝑡

Given that 𝑓 ′ = 𝜆(𝑓 − 𝑓 2 )


3
̅̅̅ = [0.5
𝑛𝑒𝑡1 = 𝑊1𝑇 ∗ 𝑥1 1 ]
−1 2 ]=4.5
[
−1
1
𝑂1 = 𝑓(𝑛𝑒𝑡1) =
1 − 𝑒 −4.5
= 0.989, 𝑑𝑖 ≠ 𝑂𝑖

Therefore we obtain updated weight vector

𝑓 ′ (𝑛𝑒𝑡1) = (𝑂1 − (𝑂1)2 = (0.989 − (0.989)2 ) = 0.01086623

Δ𝑤1 = 𝐶 ∗ (𝑑1 − 𝑂1) ∗ 𝑓 ′ (𝑛𝑒𝑡1)*X1

3 0.0000537
Δ𝑤1 = 0.15 ∗ (1 − 0.989) ∗ 0.01086623 ∗ [ 2 ] = [ 0.0000358 ]
−1 −0.0000179
0.5 0.0000537 0.5000537
𝑊2 = 𝑊2 + Δ𝑤1 = [ 1 ] + [ 0.0000358 ] = [ 1.0000358 ]
−1 −0.0000179 −1.0000179

Step2: Second sample X2

−1
𝑛𝑒𝑡2 = 𝑊2𝑇 ∗ 𝑋2 = [0.5000537 1.0000358 −1.0000179] [ 0 ]= -2.5
2
1
𝑂2 = 𝑓 (𝑛𝑒𝑡2) = = 0.076 𝑑𝑖 ≠ 𝑂𝑖
1 − 𝑒 −(−2.5)
Therefore we obtain updated weight vector

𝑓 ′ (𝑛𝑒𝑡2) = (𝑂 − (𝑂2)2 ) = (0.076 − (0.076)2 ) = 0.07009839


Δ𝑤2 = 𝐶 ∗ (𝑑2 − 𝑂2) ∗ 𝑓 ′ (𝑛𝑒𝑡2)*X2

−1 0.00754
Δ𝑤2 = 0.1 ∗ (−1 − 0.076) ∗0.07009839∗ [ 0 ] = [ 0 ]
2 −0.0151
0.5000537 0.00754 0.5076
𝑊3 = 𝑊2 + Δ𝑤2 = [ 1.0000358 ] + [ 0 ] = [1.0000358]
−1.0000179 −0.0151 −1.0150

_____________________________________________________________

Q4: Train a neural network with Delta Rule algorithm using the following:

3 −1 0.5
̅̅̅ ̅̅̅ ̅̅̅̅̅
𝑥1 = [ 2 ], 𝑥2 = [ 0 ], initial weights vector is 𝑊1 = [0.4].
−1 2 −1
C=0.15, λ=1, d1= -1, d2=1 . Use continuous bipolar activation function
2
𝑓 (𝑛𝑒𝑡) = −1 (15 marks)
1+𝑒 −𝜆 𝑛𝑒𝑡
3
𝑛𝑒𝑡1 = 𝑊1𝑇 ∗ 𝑋1 = [0.5 ]
0.4 −1 2 ]=3.3
[
−1
2
𝑂1 = 𝑓(𝑛𝑒𝑡1) = − 1 = 0.929, 𝑑𝑖 ≠ 𝑂𝑖
1 − 𝑒 −3.3
Therefore we obtain updated weight vector
1 1
𝑓 ′ (𝑛𝑒𝑡1) = (1 − (𝑂1)2 = (1 − (0.929)2 ) =0.06861176
2 2

Δ𝑤1 = 𝐶 ∗ (𝑑1 − 𝑂1) ∗ 𝑓 ′ (𝑛𝑒𝑡1)*X1

3 −0.059554
Δ𝑤1 = 0.15 ∗ (−1 − 0.929) ∗ 0.06861176 ∗ [ 2 ] = [ −0.03970 ]
−1 0.0198513
0.5 −0.059554 0.44044
𝑊2 = 𝑊1 + Δ𝑤1 = 0.4 + −0.03970 = 0.360 ]
[ ] [ ] [
−1 0.0198513 −0.980

Step2: Second sample X2

−1
𝑇 [ ]
𝑛𝑒𝑡2 = 𝑊2 ∗ 𝑋2 = 0.44044 0.360 −0.980 [ 0 ]= -2.4
2
2
𝑂2 = 𝑓 (𝑛𝑒𝑡2) = − 1 = −0.834, 𝑑𝑖 ≠ 𝑂𝑖
1 − 𝑒 −(−2.4)
Therefore we obtain updated weight vector
1 1
𝑓 ′ (𝑛𝑒𝑡2) = 2 (1 − (𝑂2)2 ) = 2 (1 − (−0.834)2 ) = 0.15241552

Δ𝑤2 = 𝐶 ∗ (𝑑2 − 𝑂2) ∗ 𝑓 ′ (𝑛𝑒𝑡2)*X2


−1 −0.0419
Δ𝑤2 = 0.15 ∗ (1 − (−0.834)) ∗ 0.15241552 ∗ [ 0 ] = [ 0 ]
2 0.08385
−0.44044 −0.0419 0.398
𝑊3 = 𝑊2 + Δ𝑤2 = [ 0.360 ] + [ 0 ] = [ 0.36 ]
−0.980 0.08385 −0.896

_____________________________________________________________

Q5: Apply the Delta Rule algorithm to train a network given the following:
0.5
̅̅̅̅̅
The initial weights vector is 𝑊1 = [0.2]. While the input patterns are
1
2 −1
̅̅̅ ̅̅̅
𝑥1 = [ 3 ], 𝑥2 = [ 0 ], and C=0.15, λ=1, d1= -1, d2=1 . Use continuous
−1 2
2
bipolar activation function 𝑓 (𝑛𝑒𝑡) = −𝜆 𝑛𝑒𝑡
(15 marks)
1+𝑒

2
𝑇
𝑛𝑒𝑡1 = 𝑊1 ∗ 𝑋1 = [0.5 0.2 1] [ 3 ]=0.6
−1
2
𝑂1 = 𝑓(𝑛𝑒𝑡1) = − 1 = 0.2913, 𝑑𝑖 ≠ 𝑂𝑖
1 − 𝑒 −0.6
Therefore we obtain updated weight vector
1 1
𝑓 ′ (𝑛𝑒𝑡1) = 2 (1 − (𝑂1)2 = 2 (1 − (0.2913)2 ) =0.45756848
Δ𝑤1 = 𝐶 ∗ (𝑑1 − 𝑂1) ∗ 𝑓 ′ (𝑛𝑒𝑡1)*X1

2 −0.177
Δ𝑤1 = 0.15 ∗ (−1 − 0.2913) ∗ 0.45756848 ∗ [ 3 ] = [−0.266]
−1 0.0886
0.5 −0.177 0.322
𝑊2 = 𝑊1 + Δ𝑤1 = [0.2] + [−0.266] = [−0.066]
1 0.0886 1.0886

Step2: Second sample X2

−1
𝑇 [ ]
𝑛𝑒𝑡2 = 𝑊2 ∗ 𝑋2 = 0.322 −0.066 1.0886 [ 0 ]= 1.85
2
2
𝑂2 = 𝑓 (𝑛𝑒𝑡2) = − 1 = 0.73, 𝑑𝑖 ≠ 𝑂𝑖
1 − 𝑒 −(1.85)
Therefore we obtain updated weight vector
1 1
𝑓 ′ (𝑛𝑒𝑡2) = 2 (1 − (𝑂2)2 ) = 2 (1 − (0.73)2 ) = 0.23405093

Δ𝑤2 = 𝐶 ∗ (𝑑2 − 𝑂2) ∗ 𝑓 ′ (𝑛𝑒𝑡2)*X2

−1 −0.0095
Δ𝑤2 = 0.15 ∗ (1 − (0.73)) ∗ 0.23405093 ∗ [ 0 ] = [ 0 ]
2 0.019
0.322 −0.0095 0.313
𝑊3 = 𝑊2 + Δ𝑤2 = [−0.066] + [ 0 ] = [ −0.066]
1.0886 0.019 1.11

________________________________________________________________________

Q6: For a network with 3 inputs and initial weights of 0.5, 0.2 and 1 use
3 −1
̅̅̅ ̅̅̅
Delta Rule to train the network with two patterns 𝑥1 = [1], 𝑥2 = [ 0 ]
2 1
and target outputs,d1= -1, d2=1, given that C=0.2 and λ=1 and activation
2
function of the form 𝑓(𝑛𝑒𝑡) = − 1 (15 marks)
1+𝑒 −𝜆 𝑛𝑒𝑡

3
𝑛𝑒𝑡1 = 𝑊1𝑇 ∗ 𝑋1 = [0.5 0.2 1] [1]=3.7
2
2
𝑂1 = 𝑓(𝑛𝑒𝑡1) = − 1 = 0.95, 𝑑𝑖 ≠ 𝑂𝑖
1 − 𝑒 −3.7
Therefore we obtain updated weight vector
1 1
𝑓 ′ (𝑛𝑒𝑡1) = 2 (1 − (𝑂1)2 = 2 (1 − (0.95)2 ) =0.04708982

Δ𝑤1 = 𝐶 ∗ (𝑑1 − 𝑂1) ∗ 𝑓 ′ (𝑛𝑒𝑡1)*X1


3 −0.2667
Δ𝑤1 = 0.20 ∗ (−1 − 0.95) ∗ 0.04708982 ∗ [1] = [−0.0886]
2 −0.1773
0.5 −0.2667 0.234
𝑊2 = 𝑊1 + Δ𝑤1 = [0.2] + [−0.0886] = [0.111]
1 −0.1773 0.823

Step2: Second sample X2

−1
𝑛𝑒𝑡2 = 𝑊2𝑇 ∗ 𝑋2 = [0.234 0.111 0.823] [ 0 ]= 0.59
1
2
𝑂2 = 𝑓 (𝑛𝑒𝑡2) = − 1 = 0.286, 𝑑𝑖 ≠ 𝑂𝑖
1 − 𝑒 −(0.59)
Therefore we obtain updated weight vector
1 1
𝑓 ′ (𝑛𝑒𝑡2) = 2 (1 − (𝑂2)2 ) = 2 (1 − (0.286)2 ) = 0.459

Δ𝑤2 = 𝐶 ∗ (𝑑2 − 𝑂2) ∗ 𝑓 ′ (𝑛𝑒𝑡2)*X2

−1 −0.0095
Δ𝑤2 = 0.2 ∗ (1 − (0.286)) ∗ 0.459 ∗ [ 0 ] = [ 0 ]
1 0.0095
0.234 −0.0095 0.2246
𝑊3 = 𝑊2 + Δ𝑤2 = [0.111] + [ 0 ] = [0.1114]
0.823 0.0095 0.8322
_____________________________________________________________

_____________________________________________________________

Q15: Can delta rule be applied to a network with identity activation


function? (5 marks)

Answer: yes, because identity function is continuous

_____________________________________________________________

Q16: Can delta rule be applied to a network with step function activation? (5
marks)

Answer: no, because step function is non-continuous

Q17: Delta rule can be applied to a______________ activation function. (5


marks)

Answer: continuous

You might also like