Neural Networks & Fuzzy Systems

OBJECTIVES

 Biological motivation to design intelligent systems and control

 Study the learning strategies of Artificial Neural networks and their

training algorithms.

 Discuss neural networks, architectures, algorithms and applications,

including Back-propagation, adaptive networks, BAM, Hopfield network

 To acquire knowledge about associate memory and training algorithms of

various associate memory networks.

 Introducing the concepts of classical and fuzzy set theory in crisp and
fuzzy logic principle, fuzziness involved in various systems

 Study the fuzzy rule base system, decision making system and different
methods of defuzzification and applications of fuzzy logic
Page 18 of 287
 Neural Networks & Fuzzy Systems
OUTCOMES
 To differentiate Biological system, intelligent systems and the
concepts of crisp and fuzzy set theory
 To analyze the learning strategies of Artificial Neural networks
and learning rules
 To understand training algorithms and are able to provide
adequate knowledge about feed forward and feedback neural
networks.
 To design training algorithms for associative memory network for
pattern recognition problems
 To demonstrate knowledge and understanding of fuzzy system as
they apply in real time systems
 To apply different methodologies to solve the problems related to
defuzzification.

Page 19 of 287
 Course Outcomes & Program Outcomes Mapping

PO12
PO11

PSO2

PSO3
PSO1
PO10
PO7
PO1

PO2

PO3

PO4

PO5

PO6

PO8

PO9
CO/PO

CO1 3 3 3 2 1 - - - - - - - 1 - -
CO2 3 3 2 2 2 - - - - - - - 1 - -
CO3 3 3 3 3 2 - - - - - - 1 1 - -
CO4 3 3 3 2 3 - 2 - - - - 2 1 - -
CO5 3 3 3 2 1 - - - - - - - 1 - -
CO6 3 3 3 3 2 - - - - - - 2 1 - -

Page 20 of 287
 NN & FS Syllabus
UNIT I ARTIFICIAL NEURAL NETWORKS
Introduction, Biological Neuron, Artificial Neuron, Basic concepts of Neural
Networks, Basic Models of ANN Connections, McCulloch-Pitts Model,
Characteristics of ANN, Applications of ANN

UNIT II ESSENTIALS OF ARTIFICIAL NEURAL NETWORKS

Artificial Neuron Model, Operations of Artificial Neuron, Types of Neuron
Activation Function, ANN Architectures, Classification Taxonomy of ANN –
Connectivity, Learning Strategies (Supervised, Unsupervised, Reinforcement),
Learning Rules, Numerical problems, Types of Application

UNIT III SUPERVISED LEARNING NETWORKS

Perceptron Network, Perceptron Learning Rule, Architecture, Perceptron
Training Algorithm, ADALINE, MADALINE, Back Propagation Network, BP
Learning Rule, Input Layer Computation, Hidden Layer Computation, Output
Layer Computation, Radial Basis Function Demonstration through MATLAB
Page 21 of 287
 NN & FS Syllabus
UNIT IV ASSOCIATIVE MEMORY NETWORK
Training Algorithms for Pattern Association, Auto Associative Memory
Network, Hetero Associative Memory Network, BAM, Hopfield
Networks, Applications

UNIT V CLASSICAL & FUZZY SETS

Introduction to classical sets - properties, Operations and relations;
Fuzzy sets, Membership, Uncertainty, Operations, properties, fuzzy
relations, cardinalities, membership functions.

UNIT VI FUZZY LOGIC SYSTEM COMPONENTS

Fuzzification, Membership value assignment, development of rule
base and decision making system, Defuzzification to crisp sets,
Defuzzification methods, Applications Page 22 of 287
 NN & FS References
TEXT BOOKS:
1. Neural Networks and Fuzzy Logic System by Bart
Kosko, PHI Publications.
2. Neural Networks, Fuzzy logic, Genetic algorithms:
synthesis and applications by Rajasekharan and Pai –
PHI Publications.
3. Fundamental of Artificial Neural Network and Fuzzy
Logic-by Rajesh Kumar, Lakshmi publications

REFERENCE BOOKS:
1. Neural Networks – James A Freeman and Davis
Skapura, Pearson Education.
2. Neural Networks – Simon Hakins , Pearson Education

Page 23 of 287
 Presentation Outline
 Need of Controller
 Types of Controllers
 Human Vs Computer
 Artificial Intelligence
 Biological Neuron & Artificial Neural Network
 Basic Concepts of ANN
 Learning Strategies & Rules
 Perceptron Networks
 Single Layer Networks
 Multi Layer Networks
 Back Propagation Algorithm
 Associative Memories
Page 24 of 287
 Applications
Need of Controller

Page 25 of 287
Need of Controller

Page 26 of 287
 Type of Controllers
 Proportional Controller
 Integral Controller
 Differential Controller
 Proportional & Integral Controller
 Proportional & Differential Controller
 Proportional, Integral & Differential
Controller
 Hysteresis Controller
 Neural Controller
 Fuzzy Controller etc.,

Page 27 of 287
Human Vs Computer

Page 28 of 287
Human Vs Computer

Page 29 of 287
 Human Vs Computer
Specialized
General purpose State based Physical modelling
numerical
languages simulation environment
mathematics
Ex. FORTRAN Ex. Simulink Ex. MapleSim
Ex. MATLAB

Problem Analysis Problem Analysis Problem Analysis Problem Analysis

Effort

Intuition &
Intuition & Physics Intuition & Physics Intuition & Physics

Computer Effort
Physics
Model Equations Model Equations Model Equations Model Equations
Human

Simulation Model Simulation Model Simulation Model Simulation Model

Numerical Numerical Numerical Numerical
Algorithm Algorithm Algorithm Algorithm
Execute Numerical Execute Numerical Execute Numerical Execute Numerical
Algorithms Algorithms Algorithms Algorithms
Mathematical
Numerical experts Modelling experts Engineers
experts

Mathematical experts Modelling experts Engineers

Modelling experts Engineers

Page 30 of 287
Engineers
Organisation of Human Brain

Page 31 of 287
Information flow in nervous system
 Biological Neuron
Study of Neural Networks originates in biological
systems
Human Brain contains over 1011 number of neurons
 Each neuron connects to 104 number of synapses
switching time of a neuron is order of milliseconds

2.5 million
gigabytes of
storage
Page 32 of 287
Page 33 of 287
Biological Neuron

Page 34 of 287
Biological Neuron

Page 35 of 287
Biological Neuron Vs Artificial Neuron

Page 36 of 287
 Artificial Neuron Model

Netj = x1w1j
=
+x2w2j+……+xnw1n
oj = φ (Netj - θj)
Page 37 of 287
 Characteristics of NN
 The NNs exhibit mapping capabilities, i.e. they can
map input patterns to their associated output patterns
 The NNs learn by examples, i.e. NN architectures
can be trained with known examples of a problem
before they are tested for their inference capability on
unknown instances of the problem. Therefore they can
identify previously untrained.
The NNs possess the capability to generalize, i.e. they
can predict new outcomes from past trends.
The NNs are robust systems and fault tolerant, i.e.
they can recall full patterns from incomplete, partial or
noisy patterns.
The NNs can process information in parallel, at high
speed and in a distributed manner. Page 38 of 287
 The McCulloch-Pitts Neuron
The activation of a McCulloch Pitts neuron is binary.

Neurons are connected by directed weighted paths.

A connection path is excitatory if the weight on the path

is positive else its inhibitory.

All excitatory connections to a neuron have the same

weights.
Each neuron has a fixed threshold:

f(n) = 1 if n ≥ θ
0 if n < θ
The threshold is set so that inhibition is absolute.
Page 39 of 287
The McCulloch-Pitts Neuron

 n

1

if  i i T
w x k

o
i 1
k+1 =  n
0

if  i i T
w x
i 1
k

Page 40 of 287
 Examples

 Train a McCulloch-Pitts neural network

to perform the OR function.
 Train a McCulloch-Pitts neural network
to perform the AND function.
 Train a McCulloch-Pitts neural network
to perform the AND NOT function.
 Train a McCulloch-Pitts neural network to
perform the XOR function.

Page 41 of 287