FACULTY OF INFORMATION & COMMUNICATION

TECHNOLOGY
UNIVERSITI TEKNIKAL MALAYSIA MELAKA

BITI 3133 NEURAL NETWORK

ASSIGNMENT 2 SECTION B

STUDENT’S : NUR SYUHADA BINTI AZHAR

NAME
MATRIC NO. : B032010378
SECTION : S1G2

LECTURER : PROFESSOR TS. DR. BURHANUDDIN BIN MOHD

ABOOBAIDER
QUESTION:
Assume a standard 3-2-2 three-layered network with η = 1.1 and α= 0.6. Further assume that
intitial values for all biases are 0.005, the values of input to hidden weights are -0.27 and hidden
to output weights are 0.31. Perform calculations using backpropagation learning for two
iterations on the pattern sets as given:

k 𝒙𝒌𝟏 𝒙𝒌𝟐 𝒙𝒌𝟑 𝒅𝒌𝟏 𝒅𝒌𝟐

1 0.5 0.1 0.3 1 0
2 0.4 0.6 0.4 0 1

Pattern 1: Given η = 1.1 and momentum, α= 0.6.

0.005 0.005
0.5

0.31

0.1

0.31

0.3
0.005 0.005

Forward pass on pattern 1:

z11 = [x11 × m111 ] + [x12 × m121 ] + [x13 × m131 ] + m101

= [0.5 x (−0.27)] + [0.1 x (−0.27)] + [0.3 x (−0.27)] + 0.005

= (−0.238)

z21 = [x11 × m112 ] + [x12 × m122 ] + [x13 × m132 ] + m102

= [0.5 x (−0.27)] + [0.1 x (−0.27)] + [0.3 x (−0.27)] + 0.005

= (−0.238)
 1
S(z11 ) = [ 1 ]
1+e−z1

1
= [1+e0.238]

= 0.4408

1
S(z21 ) = [ 1 ]
1+e−z2

1
= [1+e0.238]

= 0.4408

y11 = [S(z21 ) × n111 ] + [S(z21 ) × n121 ] + n101

= [0.4408 × 0.31] + [0.4408 × 0.31] + 0.005

= 0.2783

y21 = [S(z21 ) × n112 ] + [S(z21 ) × n122 ] + n102

= [0.4408 × 0.31] + [0.4408 × 0.31] + 0.005

= 0.2783

1
S(y11 ) = [ 1 ]
1+e−y1

1
= [1+e−0.2783 ]

= 0.5691

1
S(y21 ) = [ 1 ]
1+e−y2

1
= [1+e−0.2783 ]

= 0.5691

e11 = d11 − S(y11 )

= 1 – 0.5691

= 0.4309
e12 = d12 − S(y21 )

= 0 – 0.5691

= (-0.5691)

δ11 = e11 × S(z11 )[1 − S(z11 )]

= 0.4309 × 0.4408(1 − 0.4408)

= 0.1062

δ12 = e12 × S(z21 )[1 − S(z21 )]

= −0.5691 × 0.4408(1 − 0.4408)

= (−0.1403)

δH11 = [(δ11 × n111 ) + (δ12 × n112 )] × [S(z11 )[1 − S(z11 )]]

= [(0.1062 × 0.31) + (−0.1403 × 0.31)] × [0.4408[1 − 0.4408]]

= (−0.0026)

δH21 = [(δ11 × n121 ) + (δ12 × n122 )] × [S(z21 )[1 − S(z21 )]]

= [(0.1062 × 0.31) + (−0.1403 × 0.31)] × [0.4408[1 − 0.4408]]

= (−0.0026)

Weight changes are affected with η = 1.1;

∆m101 = η × δH11 × 1

= 1.1 × (−0.0026) × 1

= (−0.0029)

∆m111 = η × δH11 × S(x11 )

= 1.1 × (−0.0026) × 0.5

= (−0.0014)
∆m121 = η × δH11 × S(x12 )

= 1.1 × (−0.0026) × 0.1

= (−0.0003)
∆m131 = η × δH11 × S(x13 )

= 1.1 × (−0.0026) × 0.3

= (−0.0009)

∆m102 = η × δH21 × 1

= 1.1 × (−0.0028) × 1

= (−0.0031)

∆m112 = η × δH21 × S(x11 )

= 1.1 × (−0.0026) × 0.5

= (−0.0014)

∆m122 = η × δH21 × S(x12 )

= 1.1 × (−0.0026) × 0.1

= (−0.0003)

∆m132 = η × δH21 × S(x12 )

= 1.1 × (−0.0026) × 0.3

= (−0.0009)

∆n101 = η × δ11 × 1

= 1.1 × 0.1062 × 1

= 0.1168

∆n111 = η × δ11 × S(z11 )

= 1.1 × 0.1062 × 0.4408

= 0.0515
∆n121 = η × δ11 × S(z21 )

= 1.1 × 0.1062 × 0.4408

= 0.0515

∆n102 = η × δ12 × 1

= 1.1 × (−0.1403) × 1

= (−0.1543)

∆n112 = η × δ12 × S(z11 )

= 1.1 × (−0.1403) × 0.4408

= (−0.0680)

∆n122 = η × δ12 × S(z21 )

= 1.1 × (−0.1403) × 0.4408

= (−0.0680)

Weights are modified;

n201 = 𝑛101 + ∆𝑛101

= 0.005 + 0.1168

= 0.1218

2 1 1
n11 = 𝑛11 + ∆𝑛11

= 0.31 + 0.0515

= 0.3615

n221 = 𝑛121 + ∆𝑛121

= 0.31 + 0.0515

= 0.0565
n202 = 𝑛102 + ∆𝑛102

= 0.005 + (−0.1543)

= (−0.1493)

2 1 1
n12 = 𝑛12 + ∆𝑛12

= 0.31 + (−0.0680)

= 0.242

n222 = 𝑛122 + ∆𝑛122

= 0.31 + (−0.0680)

= 0.242

2
𝑚01 = 𝑚101 + ∆𝑚101

= 0.005 + (−0.0029)

= (0.0021)

2 1 1
𝑚11 = 𝑚11 + ∆𝑚11

= (−0.27) + (−0.0014)

= (−0.2714)

2
𝑚21 = 𝑚121 + ∆𝑚121

= (−0.27) + (−0.0003)

= (−0.2703)

2
𝑚31 = 𝑚131 + ∆𝑚131

= (−0.27) + (−0.0009)

= (−0.2709)

2
𝑚02 = 𝑚102 + ∆𝑚102

= 0.005 + (−0.0029)

= (0.0021)
 2 1 1
𝑚12 = 𝑚12 + ∆𝑚12

= (−0.27) + (−0.0014)

= (−0.2714)

2
𝑚22 = 𝑚122 + ∆𝑚122

= (−0.27) + (−0.0003)

= (−0.2703)

2
𝑚32 = 𝑚132 + ∆𝑚132

= (−0.27) + (−0.0009)

= (−0.2709)

Notice that since this is the first iteration there is no effect of the momentum term since the
weight change on the previous iteration was zero.

Pattern 2:

0.0021 0.1218
0.4

0.3615

0.6

0.242

0.4
0.0021 -0.1493

Forward pass on pattern 2 :

z12 = [x12 × m11

2 ]
+ [x22 × m221 ] + [x32 × m231 ] + m201

= [0.4 × (−0.2714)] + [0.6 × (−0.2703)] + [0.4 × (−0.2709)] + 0.0021

= (-0.377)
z22 = [x12 × m12
2 ]
+ [x22 × m222 ] + [x32 × m232 ] + m202

= [0.4 × (−0.2714)] + [0.6 × (−0.2703)] + [0.4 × (−0.2709)] + 0.0021

= (-0.377)

1
S(z12 ) = [ 2 ]
1+e−z1

1
= [1+e0.377]

= 0.4069

1
S(z22 ) = [ 2 ]
1+e−z1

1
= [1+e0.377]

= 0.4069

y12 = [S(z12 ) × n11

2 ]
+ [S(z12 ) × n221 ] + n201

= [0.4069 × 0.3615] + [0.4069 × 0.3615] + 0.1218

= 0.4160

y22 = [S(z22 ) × n12

2 ]
+ [S(z22 ) × n222 ] + n202

= [0.4069 × 0.242] + [0.4069 × 0.242] + (−0.1493)

= 0.0476

1
S(y12 ) = [ 2 ]
1+e−y1

1
= [1+e−0.4160 ]

= 0.6025

1
S(y22 ) = [ 2 ]
1+e−y2

1
= [1+e−0.0476 ]

= 0.5119
e12 = d12 − S(y12 )

= 0 – 0.6025

= (-0.6025)

e22 = d22 − S(y22 )

= 1 – 0.5119

= 0.4881

δ12 = e12 × S(z12 )[1 − S(z12 )]

= (−0.6025 × 0.4069(1 − 0.4069)

= (−0.1454)

δ22 = e22 × S(z22 )[1 − S(z22 )]

= 0.4881 × 0.4069(1 − 0.4069)

= 0.1178

δH12 = [(δ12 × n11

2 )
+ (δ22 × n12
2 )]
× [S(z12 )[1 − S(z12 )]]

= [(−0.1454 × 0.3615) + (0.1178 × 0.242)] × [0.4069[1 − 0.4069]]

= (−0.0457)

δH22 = [(δ12 × n221 ) + (δ22 × n222 )] × [S(z22 )[1 − S(z22 )]]

= [(−0.1454 × 0.3615) + (0.1178 × 0.242)] × [0.4069[1 − 0.4069]]

= (−0.0457)

Weight changes are affected with η = 1.1;

∆m201 = η × δH12 × 1

= 1.1 × (−0.0457) × 1

= (−0.0503)
 2
∆m11 = η × δH12 × S(x12 )

= 1.1 × (−0.0457) × 0.4

= (−0.0201)

∆m221 = η × δH12 × S(x22 )

= 1.1 × (−0.0457) × 0.6

= (−0.0302)

∆m231 = η × δH12 × S(x32 )

= 1.1 × (−0.0457) × 0.4

= (−0.0201)

∆m202 = η × δH22 × 1

= 1.1 × (−0.0457) × 1

= (−0.0503)

2
∆m12 = η × δH22 × S(x12 )

= 1.1 × (−0.0457) × 0.4

= (−0.0201)

∆m222 = η × δH22 × S(x22 )

= 1.1 × (−0.0457) × 0.6

= (−0.0302)

∆m232 = η × δH22 × S(x22 )

= 1.1 × (−0.0457) × 0.4

= (−0.0201)

∆n201 = η × δ12 × 1

= 1.1 × (−0.1454) × 1

= (−0.1599)
 2
∆n11 = η × δ12 × S(z12 )

= 1.1 × (−0.1454) × 0.4069

= (−0.0651)

∆n221 = η × δ12 × S(z22 )

= 1.1 × (−0.1454) × 0.4069

= (−0.0651)

∆n202 = η × δ22 × 1

= 1.1 × 0.1178 × 1

= 0.1296

2
∆n12 = η × δ22 × S(z12 )

= 1.1 × 0.1178 × 0.4069

= 0.0527

∆n222 = η × δ22 × S(z22 )

= 1.1 × 0.1178 × 0.4069

= 0.0527

Weights are modified;

Since there were weights changes in the previous iteration, the momentum term comes into
the update procedure.

n301 = n201 + ∆n201 + ∝ × ∆n101

= 0.1218 + (−0.1599) + 0.6 × 0.1168

= 0.0320

3 2 2
n11 = n11 + ∆n11 + ∝ × ∆n111

= 0.3615 + (−0.0651) + 0.6 × 0.0515

= 0.3273
n321 = n221 + ∆n221 + ∝ × ∆n121

= 0.3615 + (−0.0651) + 0.6 × 0.0515

= 0.3273

n302 = n202 + ∆n202 + ∝ × ∆n102

= (−0.1493) + 0.1296 + 0.6 × (−0.1543)

= (−0.1123)

3 2 2
n12 = n12 + ∆n12 + ∝ × ∆n112

= 0.242 + 0.0527 + 0.6 × (−0.0680)

= 0.2539

n322 = n222 + ∆n222 + ∝ × ∆n122

= 0.242 + 0.0527 + 0.6 × (−0.0680)

= 0.2539

m301 = m201 + ∆m201 + ∝ × ∆m101

= 0.0021 + (−0.0503) + 0.6 + (−0.0029)

= (−0.0499)

3 2 2
m11 = m11 + ∆m11 + ∝ × ∆m111

= (−0.2714) + (−0.0201) + 0.6 × (−0.0014)

= (−0.2923)

m321 = m221 + ∆m221 + ∝ × ∆m121

= (−0.2703) + (−0.0302) + 0.6 × (−0.0003)

= (−0.3007)

m331 = m231 + ∆m231 + ∝ × ∆m131

= (−0.2709) + (−0.0201) + 0.6 × (−0.0009)

= (−0.2915)
m302 = m202 + ∆m202 + ∝ × ∆m102

= 0.0021 + (−0.0503) + 0.6 + (−0.0029)

= (−0.0499)

3 2 2
m12 = m12 + ∆m12 + ∝ × ∆m112

= (−0.2714) + (−0.0201) + 0.6 × (−0.0014)

= (−0.2923)

m322 = m222 + ∆m222 + ∝ × ∆m122

= (−0.2703) + (−0.0302) + 0.6 × (−0.0003)

= (−0.3007)

m332 = m232 + ∆m232 + ∝ × ∆m132

= (−0.2709) + (−0.0201) + 0.6 × (−0.0009)

= (−0.2915)

Pattern 3:

-0.0499 0.0320

0.3273

0.2539

-0 .0499 -0.1123