BEE4333 INTELLIGENT

CONTROL
Chapter 4 :
Artificial Neural Network (ANN)
Class Activity (Pop Quiz)
• Searching for 3 Winners today
• Pop Quiz.
What do you understand about Artificial
Neural Network?
What are the similarities between these
pictures?

VS
Contents
• 4.1 Basic Concept
• 4.2 ANN Applications
• 4.3 ANN Model
• 4.4 ANN Learning
• 4.5 Simple ANN
• 4.6 Multilayer Neural Networks &
Backpropagation Algorithm
Basic Concept
• ANN born from the demand of machine learning;
computer learns from experience, examples and
analogy.
• Simple concept : computer attempts to model the
human brain.
• Also known as parallel distributed processors.
• Why we need an intelligent processor or computer
to replace current technology?
– To decide intelligently and interact accordingly.
Human Brain; biological NN

Plasticity : Neurons heading to right answer are Learning from

strengthen and for the wrong answer is weakened. experience!
Learning

OUTPUT SIGNALS
INPUT SIGNALS

INPUT LAYER HIDDEN LAYER OUTPUT LAYER

ANN Architecture
Learning
• Synapses has their own weight to express the
importance of input.
• The output of a neuron might be the final solution or the
input to other networks.
• ANN learns through iterated adjustment from synapses
weight.
• Weight is adjusted to cope with the output environment
regarding about its network input/output behavior.
• Each neutron computes its activation level based on the
I/O numerical weights.
How to design ANN?
 Decide how many neurons to be used.
 How the connections between neurons are
constructed? How many layers needed?
 Which learning algorithm to be apply?
 Train the ANN by initialize the weight and update
the weights from training sets.
ANN characteristics
Advantages:
• A neural network can perform tasks that a linear program can not.

• When an element of the neural network fails, it can continue without

any problem by their parallel nature.
• A neural network learns and does not need to be reprogrammed.

• It can be implemented in any application.

• It can be implemented without any problem.

Disadvantages:
• The neural network needs training to operate.

• The architecture of a neural network is different from the

architecture of microprocessors therefore needs to be emulated.
• Requires high processing time for large neural networks.
4.2
ANN APPLICATION
What application use ANN?
Upload the image here.
ANN Application
 Character recognition
 Image compression
 Stock market prediction
 Travelling salesman’s problem
 Medicine, electronic nose, security and loan
application
 Etc….
Application : Fingerprint Recognition
 INTRODUCTION
❑ Features of finger prints
❑ Finger print recognition system
❑ Why neural networks?
❑ Goal of the system
▪ Preprocessing system
▪ Feature extraction using neural networks
▪ Classification
▪ Result
Features of finger prints

Finger prints are the unique

pattern of ridges and
valleys in every person’s
fingers.
◼ Their patterns are permanent
and unchangeable for whole
life of a person.
◼ They are unique and the
probability that two
fingerprints are alike is only
1 in 1.9x10^15.
◼ Their uniqueness is used for
identification of a person.
 Finger print recognition system

Image Edge Ridge Thinning Feature Classifi

acquisition detection extraction extraction cation

 Image acquisition: the acquired image is digitalized into 512x512

image with each pixel assigned a particular gray scale value
(raster image).
 edge detection and thinning: these are preprocessing of the image ,
remove noise and enhance the image.
 Finger print recognition system
 Feature extraction: this the
step where we point out
the features such as ridge
bifurcation and ridge
endings of the finger print
with the help of neural
network.
 Classification: here a class
label is assigned to the
image depending on the
extracted features.

Image Edge Ridge Thinning Feature Classifi

acquisition detection extraction extraction cation
Why using neural networks?
 Neural networks enable us to find solution where
algorithmic methods are computationally intensive or
do not exist.
 There is no need to program neural networks they
learn with examples.
 Neural networks offer significant speed advantage
over conventional techniques.
Preprocessing system
The first phase of finger print recognition is to capture
a image .
◼ The image is captured using total internal reflection of light (TIR).
◼ The image is stored as a two dimensional array of 512x512 size,
each element of array representing a pixel and assigned a gray
scale value from 256 gray scale levels.
 Preprocessing system
After image is captured ,noise is removed using
edge detection, ridge extraction and thinning.
◼ Edge detection: the edge of the image is defined
where the gray scale levels changes greatly. also,
orientation of ridges is determined for each 32x32
block of pixels using gray scale gradient.
◼ Ridge extraction: ridges are extracted using the fact
that gray scale value of pixels are maximum along the
direction normal to the ridge orientation.
 Preprocessing system
 Thinning: the extracted ridges are
converted into skeletal structure in which
ridges are only one pixel wide. thinning
should not-
◼ Remove isolated as well as surrounded
pixel.
◼ Break connectedness.
◼ Make the image shorter.
 Feature extraction using neural
networks
 Multilayer perceptron network of
three layers is trained to detect
minutiae in the thinned image.
◼ The first layer has nine perceptrons
◼ The hidden layer has five perceptrons
◼ The output layer has one perceptron.
The network is trained to output ‘1’
when the input window is centered
at the minutiae and it outputs ‘0’
when minutiae are not present.
Feature Extraction Using Neural
Networks
 Trained neural networks
are used to analyze the
image by scanning the
image with a 3x3
window.
 To avoid falsely reported
features which are due to
noise –
◼ The size of scanning
window is increased to
5x5
◼ If the minutiae are too
close to each other than
we ignore all of them.
Classification

finger prints can be classified

mainly in four classes
depending upon their
general pattern-
◼ Arch
◼ Tented arch
◼ Right loop
◼ Left loop
Applications of Fingerprint Recognition

As finger print recognition system can be easily

embedded in any system. It is used in-
◼ Recognition of criminals in law enforcement bodies.
◼ Used to provide security to cars, lockers, banks ,shops.
◼ To differentiate between a person who has voted and those who
have not voted in govt. elections.
◼ To count individuals.
4.3
ANN MODEL
ANN Models
 Feed-forward networks
 Radial basis function networks
 Recurrent networks
 Echo state networks
 Hopfield networks
 Competitive model
 Self-organizing maps
 ART model
 Boltzmann machine
 Committee of machines
 Etc….
 Feed-forward Network

x1 w1 y1

x2 w2 y2

x3 w3 y3

http://en.wikibooks.org/wiki/Artificial_Neural_Networks/Feed-Forward_Networks
 Radial Basis Function Networks
❑ Each hidden layer neuron represents
a basis function of the output space,
with respect to a particular center in
the input space.
❑ The activation function chosen is
commonly a Gaussian kernel:

 3 layers ❑ This kernel is centered at the point in

 Hidden layer : Radial basis activation the input space specified by the
function (RBF) weight vector. The closer the input
 Output layer of linear summation unit(s). signal is to the current weight vector,
 Only the tap weights between the hidden the higher the output of the neuron
layer and the output layer are modified will be. Radial basis function
during training.
networks are used commonly in
function approximation and series
http://bio.felk.cvut.cz/biocmsms/index.php?page=neural-networks
prediction.
Recurrent Networks
❑ In contrast to feed-forward networks, it has feedback
elements that enable signals from one layer to be fed back
to a previous layer.
❑ A simple recurrent network has three layers, an input, an
output, and a hidden layer.
❑ A set of additional context units are added to the input
layer that receive input from the hidden layer neurons.
❑ The feedback paths from the hidden layer to the context
units have a fixed weight of unity.
❑ A fully recurrent network is one where every neuron
receives input from all other neurons in the system. Such
networks cannot be easily arranged into layers.
❑ A small subset of neurons receives external input, and
another small subset produce system output.

http://en.wikibooks.org/wiki/Artificial_Neural_Networks/Recurrent_Networks
 Echo State Networks
 Recurrent networks where the hidden layer
neurons are not completely connected to all
input neurons.
 Known as sparsely connected networks.
 Only the weights from the hidden layer to the
output layer may be altered during training.
 Echo state networks are useful for matching
and reproducing specific input patterns.
Because the only tap weights modified during
training are the output layer tap weights.
 Training is typically quick and computationally
efficient in comparison to other multi-layer
networks that are not sparsely connected.

http://www.scholarpedia.org/article/Echo_state_network
Hopfield Networks

 Competitive Networks

- Competitive networks are

networks where neurons
compete with one another.
- The weight vector is treated as a
"prototype", and is matched
against the input vector.
- The "winner" of each training
session is the neuron whose
weight vector is most similar to
the input vector.
http://en.wikibooks.org/wiki/Artificial_Neural_Networks/Competitive_Models
4.4
ANN LEARNING
ANN Learning
• In all of the neural paradigms, the application of
an ANN involves two phases:
– (1) Learning phase
– (2) Testing phase
• In the learning phase (usually offline) the ANN is
trained until it has learned its tasks (through the
adaptation of its weights)
• The Testing phase is used to solve the task.
ANN Learning
• An ANN solves a task when its weights are adapted through
a learning phase.
• All neural networks have to be trained before they can be
used.
• They are given training patterns and their weights are
adjusted iteratively until an error function is minimized.
• Once the ANN has been trained no more training is needed.
• Two types of learning prevailed in ANNs:
– Supervised learning:- learning with teacher signals or targets
– Unsupervised learning:- learning without the use of teacher
signals
 Supervised Learning
• In supervised learning the training patterns are provided to
the ANN together with a teaching signal or target.
• The difference between the ANN output and the target is the
error signal.
• Initially the output of the ANN gives a large error during the
learning phase.
• The error is then minimized through continuous adaptation
of the weights to solve the problem through a learning
algorithm.
• In the end when the error becomes very small, the ANN is
assumed to have learned the task and training is stopped.
• It can then be used to solve the task in the recall phase.
Supervised Learning

Matching the I/O pattern

Unsupervised Learning
• In unsupervised learning, the ANN is trained without teaching
signals or targets.
• It is only supplied with examples of the input patterns that it will
solve eventually.
• The ANN usually has an auxilliary cost function which needs to be
minimized like an energy function, distance, etc.
• Usually a neuron is designated as a “winner” from similarities in the
input patterns through competition.
• The weights of the ANN are modified where a cost function is
minimized.
• At the end of the learning phase, the weights would have been
adapted in such a manner such that similar patterns are clustered
into a particular node.
ANN paradigm
• There are a number of ANN paradigms developed over the past
few decades.
• These ANN paradigms are mainly distinguished through their
different learning algorithms rather than their models.
• Some ANN paradigms are named after their proposer such as
Hopfield, Kohonen, etc.
• Most ANNs are named after their learning algorithm such as
Backpropagation, Competitive learning, Counter propagation, ART,
etc. and some are named after their model such as BAM.
• Basically a particular ANN can be divided into either a feedforward
or a feedback model and into either a supervised or unsupervised
learning mode.
ANN Classifications
ANN Performance
• The performance of an ANN is described by the
figure of merit, which expresses the number of
recalled patterns when input patterns are applied,
that could be complete, partially complete, or even
noisy.
• A 100% performance in recalled patterns means
that for every trained input stimulus signal, the ANN
always produces the desired output pattern.
ANN Performance
Basis of ANN computing idea
• Neuron computes the input signals and compares the result
with a threshold value, θ.
• If the input is less than θ, then the neuron output is -1,
otherwise +1.
• Hence, the following activation function(sign function) is used,

where X is the net weighted input to neuron, xi is the i input

value, wi is the weight of input i . n is the number of neuron
input and Y is the neuron output.
 Other types of activation function
Sign Sigmoid
Y
function function Y
+1 +1

0 X 0 X
-1 -1

Step Y Linear Y
function function
+1 +1

0 X 0 X
-1 -1
4.5
SIMPLE ANN
Simple ANN: A Perceptron
• Perceptron is used to classify input in
two classes; e.g class A1 or A2.
• A linear separable function is used x2
to divide the n-dimensional space as
follows; 1

0 x1
• Say, 2 inputs, then we have a
characteristics as shown on left 2
figure. θ is used to shift the bound.
• Three dimensional states is also
possible to be view.
Simple Perceptron
Must be boolean!
Inputs
x1
w1 Linear Hard
Combiner limiter Output
/bias
∑

x2 w2
θ
Threshold
Learning: Classification
• Learning is done by adjusting the actual output Y to
meet the desired output Yd.
• Usually, the initial weight is adjust between -0.5 to
0.5. At iteration k of the training example, we have the
error e as

• If the error is positive, the weight must be decrease and

otherwise must be increase.
• Perceptron learning rule also can be obtained where

α is the learning rate and 0< α<1.

Training algorithm

Example
 Consider truth table of AND operation
Input x1 Input x2 AND (x1 ∩ x2)

Threshold,
0 0 0 θ = 0.2
0 1 0 Learning rate,
α = 0.1
1 0 0
1 1 1

 How ANN of a single perceptron can be trained?

 Consider a step activation function in this example.
Epoch Input Input Desired Initial Initial Actual Error, Updated Updated
x1 x2 output yd weight weight Output e weight weight
w1(p) w2(p) Y w1(p+1) w2(p+1)

1 0 0 0 0.3 -0.1
0 1 0 0.3 -0.1
1 0 0 0.3 -0.1
1 1 1 0.2 -0.1

Actual
Output
Updated
Weight
Epoch Input Input Desired Initial Initial Actual Error, Final Final
x1 x2 output yd weight weight Output e weight weight
w1 w2 Y w1 w2

1 0 0 0 0.3 -0.1 0 0 0.3 -0.1

0 1 0 0.3 -0.1 0 0 0.3 -0.1
1 0 0 0.3 -0.1 1 -1 0.2 -0.1
1 1 1 0.2 -0.1 0 1 0.3 0.0

The epoch continues until the weights are converging to a steady

state values.
Are you ready?
• Class activity today!
• Searching for 3 winners today.
• Get ready with calculator.
Question 1

Which is True about Neuron configuration in

Neural Network?
A. Input→ Output→ Hidden Layer
B. Input→ Hidden Layer → Output
C. Hidden layer→ Input→ Output
D. Output→ Hidden Layer→ Input
Leader Board Results
Question 2

Units of one layer interact with the Neurons of

the next layer through _________.
A. Bias
B. Threshold
C. Weighted connections
D. Activation function
Leader Board Results
Question 3

Which phase does the neural networks have

to be trained before they can be used.
A. Testing phase
B. Study phase
C. Hibernate phase
D. Learning phase
Leader Board Results
Question 4
The main task for Neural Network is to find
the right _________ to get right result and
output.
A. Bias
B. Threshold
C. Weight
D. Activation function
Leader Board Results
Question 5
When should the Neural Network stop the
training process?
A. Maximum error achieved
B. Minimum error achieved
C. Correct input achieved
D. Correct hidden layer achieved
Leader Board Results
More Example AND Gate

https://www.youtube.com/watch?v=zmIzNBMsQYQ
4.6
Multilayer Neural Networks &
Backpropagation Algorithm
Are you ready?
• Class activity today!
• Searching for 3 winners today.
• Get ready with calculator.
Videos
 https://www.youtube.com/watch?v=WZDMNM36Ps
M

 https://www.youtube.com/watch?v=Ilg3gGewQ5U
Multilayer neural networks
 Multilayer NN-feedforward neural network with
one or more hidden layer.
 Model consists of input layer, middle or hidden layer
and an output layer.
 Why hidden layer is important?
 Inputlayer only receives input signal
 Output layer only display the output patterns.

 Hidden layer process the input signals; weight

represents feature of inputs.
Multilayer NN model
Inputs Output
x1

x2

x3

1st 2nd
hidden hidden
layer layer
Multilayer Neural Network Learning

 Multilayer NN learns through a learning algorithm;

the popular one is BACK-PROPAGATION.
 The computations are similar to a simple perceptron.
 Back-propagation has two phases;
 Inputlayer demonstrates the training input pattern and
then propagates from layer to layer to output.
 The calculation for error will notify the system to
modified the weights appropriately.
Back Propagation NN
 Back propagation is the learning or training
algorithm.
 Each neuron must be connected to each other.
 Sigmoid function is used for the network.
Y
+1
1
𝑌=
0 X 1 + 𝑒 −𝑋
-1
 Ideal for simple Pattern Recognition and Mapping
Tasks.
Let’s look for a case
xi wij wjk yi

n m l

inputs i j k output
Input signals
Error signals
Back Propagation
Case Study

Black pixel : 1
White Pixel : 0
Network
Training Process
 Set up the weight, all in the range of −1 <
𝑤𝑒𝑖𝑔ℎ𝑡 < 1
 Apply input pattern → calculate output. (FORWARD
PASS)
 Calculated output will be different with the TARGET.
 Differences between CALCULATED OUTPUT and
TARGET is equal to error.
 Error will be used for updating weight.
Step of Back Propagation Method
Step of Back Propagation Method (ctd)
Training Process
1) Find the neuron output in hidden layer
 𝑜𝑢𝑡𝐴 = σ31 𝑖𝑛𝑝𝑢𝑡𝑖 × 𝑤𝑖𝐴

2) Find the output of 𝛼 and 𝛽

 𝑜𝑢𝑡𝛼 = σ𝐶𝐴 𝑜𝑢𝑡𝑝𝑢𝑡𝑗 × 𝑤𝑗𝛼
 𝑜𝑢𝑡𝛽 = σ𝐶𝐴 𝑜𝑢𝑡𝑝𝑢𝑡𝑗 × 𝑤𝑗𝛽
 Training Process
3)

4)

5)

+
𝑤3𝐴 = 𝑤3𝐴 + η𝛿𝐴 𝑖𝑛𝑝𝑢𝑡3
6) +
𝑤3𝐵 = 𝑤3𝐵 + η𝛿𝐵 𝑖𝑛𝑝𝑢𝑡3
+ +
+ 𝑤3𝐶 + = 𝑤3𝐶 + η𝛿𝐶 𝑖𝑛𝑝𝑢𝑡3
𝑤1𝐴 = 𝑤1𝐴 + η𝛿𝐴 𝑖𝑛𝑝𝑢𝑡1 𝑤2𝐴 = 𝑤2𝐴 + η𝛿𝐴 𝑖𝑛𝑝𝑢𝑡2 𝑤4𝐴 = 𝑤4𝐴 + η𝛿𝐴 𝑖𝑛𝑝𝑢𝑡4
+ + +
𝑤1𝐵 = 𝑤1𝐵 + η𝛿𝐵 𝑖𝑛𝑝𝑢𝑡1 𝑤2𝐵 = 𝑤2𝐵 + η𝛿𝐵 𝑖𝑛𝑝𝑢𝑡2 𝑤4𝐵 = 𝑤4𝐵 + η𝛿𝐵 𝑖𝑛𝑝𝑢𝑡4
+ + +
𝑤1𝐶 = 𝑤1𝐶 + η𝛿𝐶 𝑖𝑛𝑝𝑢𝑡1 𝑤2𝐶 = 𝑤2𝐶 + η𝛿𝐶 𝑖𝑛𝑝𝑢𝑡2 𝑤4𝐶 = 𝑤4𝐶 + η𝛿𝐶 𝑖𝑛𝑝𝑢𝑡4
η is learning rate, usually 1.
When does the training process stop?

 Training process stop until the sum of squared error

for the output y is less than a prescribed value;
0.001.
 Sum of squared error : performance indicator of the
system.
 The smaller the error, the better the system
performance.
More about back-propagation
 Different initial weights and threshold may have
different solution, but we still have the solutions.
 The decisions boundaries can be view if we use the
sign activation function.
 Drawbacks of back-propagation
 Not suitable for biological neurons; to adjust the
neurons weight.
 Computational expensive

 Slow training
XOR PROBLEM
In this example, θj and
θk is not available!
Net j= σ𝑛𝑖=1 𝑥𝑖 𝑝 × 𝑤𝑖𝑗 𝑝 − 𝜃𝑗

Net k= σ𝑛𝑖=1 𝑥𝑖 𝑝 × 𝑤𝑖𝑗 𝑝 − 𝜃𝑘

Sigmoid activation
function
∆Wji(t+1) = 0.1 X (-0.0035) X 0
+ (0.9 X 0)
=0

Wj01(t+1) = 0.55 + (0)

= 0.55
Wj02(t+1) = 0.15 + (0)
= 0.15
Final results of Logical Operation XOR

Input Desired Actual Error Sum of

Output Output squared
x1 x2 errors
1 1 0 0.0155 -0.0155 0.0010
0 1 1 0.9849 0.0151
1 0 1 0.9849 0.0151
0 0 0 0.0175 -0.0175
Are you ready?
• Searching for 3 Winners.
• Receive bonus mark!
• Top up in Quiz, Test, Final Exam mark!
Example of Examination question
Figure 1 below illustrates a multilayer Neural Network that has the input
patterns of (0 1 1).
Sigmoid Sigmoid

W11= -0.4
θj= 0.1 θk= 0.2
0

W12= ˗0.2 W11= 0.1

1 Lk

j=1 k=1

1 W13= 0.3

Layer i Hidden Layer k

Layer j

Figure 1: A Multilayer Neural Network

i) Calculate the output value of Lk .

ii) Given tk = 0. From the value of Lk, calculate the

following values at the first iteration by using
Back Propagation algorithm.

◼ ∆w11 and w11 (new) between output and

hidden layer
◼ ∆w11, ∆w12, ∆w13, and w11(new), w12(new),
w13(new) between hidden layer and input
layer.
◼ Illustrate the new Neural Network.
The information for the Neural Networks configurations are as follows.
Given η = 0.4 and α = 0.1.
Back propagation is not required to be derived.
Sigmoid function; f(x) = (1+e-x)-1
f(x) = x
The error signals are as follows.
δk = Lk (1- Lk )( tk - Lk )
δj = Lj (1- Lj ) ∑k δk wkj
◼ Adaptions of weights are defined as below.
∆wkj( t + 1) = η δk Lj + α∆wkj( t )
∆wji( t + 1) = η δj Li + α∆wji( t )
[16 Marks]
 [CO2, P10, C4]

Question 1

What is the value of hidden layer output, 𝐿𝑗 ?

A. 0.40

B. 0.60

C. 0.50

D. 0.30
Leader Board Results
Answer
(i) Calculation of 𝐿𝑘

• Hidden Layer (𝐿𝑗 )

𝑛

𝑦𝑗 𝑝 = 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 ෍ 𝑥𝑖 𝑝 × 𝑤𝑖𝑗 𝑝 − 𝜃𝑗
𝑖=1

𝑦𝑗 𝑝 = 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 𝑥1 × 𝑊11 + 𝑥2 × 𝑊12 + 𝑥3 × 𝑊13 − 𝜃𝑗

𝑦𝑗 𝑝 = 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 0 × (−0.4) + 1 × −0.2 + 1 × 0.3 − 0.1

𝑦𝑗 𝑝 = 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 0.1 − 0.1

𝑦𝑗 𝑝 = 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 0

1
𝑦𝑗 𝑝 = 𝐿𝑗 = = 0.50
1 + 𝑒 −(0)
Question 2

What is the value of output layer, 𝐿𝑘 ?

A. 0.46

B. 0.66

C. 0.56

D. 0.36
• Output layer
𝑛

𝑦𝑘 𝑝 = 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 ෍ 𝑥𝑖 𝑝 × 𝑤𝑖𝑘 𝑝 − 𝜃𝑘
𝑖=1

𝑦𝑘 𝑝 = 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 𝑥1 × 𝑊11 − 𝜃𝑘
𝑦𝑘 𝑝 = 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 0.50 × 0.1 − 0.2
𝑦𝑘 𝑝 = 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 0.05 − 0.2
𝑦𝑘 𝑝 = 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 −0.15
𝑦𝑘 𝑝 = 𝐿𝑘 = 0.46
Question 3

What is the value of error, e=Target-Output?

A. 0.46

B. -0.66

C. -0.56

D. -0.46
Leader Board Results
Question 4

What is the value of error 𝛿𝑘 ?

A. -0.3143

B. -0.2143

C. -0.1143

D. -0.5143
Leader Board Results
Question 5

What is the value of ∆𝑤11 𝑡 + 1 at the hidden

layer ?
A. -0.0329

B. -0.0229

C. -0.0429

D. -0.0529
Leader Board Results
Question 6

What is the value of updated weight 𝑤11 ( t + 1) at

the hidden layer ?
A. 0.0771

B. 0.0871

C. 0.0971

D. 0.0671
Multiple Choice Submissions

Details can be found in the Notes section below

(i) Using back propagation method and information 𝑡𝑘 = 0 and the 𝐿𝑘 obtained
from (i).

• Calculation ∆w11 and w11 (new) between output and hidden layer

δk = Lk (1- Lk )( tk - Lk )
δk = 0.46 (1- 0.46 )(0 – 0.46 ) = -0.1143

∆𝑤11 𝑡 + 1 = η𝛿𝑘 𝐿𝑗 + 𝛼∆𝑤11 (𝑡)

∆𝑤11 ( t + 1) = (0.4)* (-0.1143)*(0.5) + (0.1)*(0)=-0.0229
𝑤11 ( t + 1) = 𝑤11 ( t ) + ∆𝑤11 ( t + 1) = (0.1) + (-0.0229) = 0.0771
Question 7

What is the value of error 𝛿𝑗 ?

A. -0.0087

B. -0.0077

C. -0.0067

D. -0.0029
Question 8

What is the value of ∆𝑤11 𝑡 + 1 and value of

updated weight 𝑤11 ( t + 1) at the input layer?

A. 0, -0.5

B. 0, -0.4

C. 0, -0.6

D. 0. -0.7
Question 9

What is the value of ∆𝑤12 𝑡 + 1 and value of

updated weight 𝑤12 ( t + 1) at the input layer?

A. 0.00116, -0.2012

B. 0.00328, -0.3023

C. 0.00428, -0.4023

D. 0.00528, -0.5023
Question 10

What is the value of ∆𝑤13 𝑡 + 1 and value of

updated weight 𝑤13 ( t + 1) at the input layer?

A. 0.00228, -0.2023

B. 0.00328, -0.3023

C. 0.00428, -0.4023

D. 0.00116, 0.2988
δj = Lj (1- Lj )σ 𝛿𝑘 𝑤𝑘𝑗
δj = 0.5 (1- 0.5 )(-0.1143*0.2) = - 0.0057

∆𝑤𝑗𝑖 (𝑡 + 1) = 𝜂𝛿𝑗 𝐿𝑖 + 𝛼∆𝑤𝑗𝑖 (𝑡)

∆𝑤11 (𝑡 + 1) = 𝜂𝛿𝑗 𝐿1 + 𝛼∆𝑤11 (𝑡) = 0.4 × −0.0057 × 0 + 0.1 × 0 = 0
∆𝑤12 (𝑡 + 1) = 𝜂𝛿𝑗 𝐿2 + 𝛼∆𝑤12 (𝑡) = 0.4 × −0.0057 × 1 + 0.1 × 0 = −2.28 × 10−3
∆𝑤13 (𝑡 + 1) = 𝜂𝛿𝑗 𝐿3 + 𝛼∆𝑤13 (𝑡) = 0.4 × −0.0057 × 1 + 0.1 × 0 = −2.28 × 10−3

𝑤11 (𝑡 + 1) = 𝑤11 (𝑡) + ∆𝑤11 (𝑡 + 1) = −0.4 + 0 = −0.4

𝑤12 (𝑡 + 1) = 𝑤12 (𝑡) + ∆𝑤12 (𝑡 + 1) = −0.2 − 2.28 × 10−3 = −0.2023
𝑤13 (𝑡 + 1) = 𝑤13 (𝑡) + ∆𝑤12 (𝑡 + 1) = 0.3 − 2.28 × 10−3 = 0.2977
New Neural Network

Sigmoid Sigmoid

W11= -0.4

θj= 0.1 θk= 0.2

0

W12= -0.2023 W11= 0.0771

1 Lk

j=1 k=1

1 W13= 0.2977

Layer i Hidden Layer j Layer k

The new neural network

 END
OF
CHAPTER 4
Quiz 3 (ANN)

 Wednesday, 18/5
 10am
 30 minutes
 KALAM Online
Assignment 2 (Individual)
 Check your Assg 1 mark in ecomm.
 Assg 2 question will be uploaded in KALAM.
 Practise tutorial for ANN.
 Due to submit? Sunday, 29 May 2022.