BCA/B.Sc.

(IT)/(CS) V Semester
Soft Computing
TBC-505/TBI-504/TBS-505
Jaishankar Bhatt
Assistant Professor
Graphic Era Deemed to be University
Dehradun
INTRODUCTION TO SOFT COMPUTING

► Concept of Computing
► Hard Computing
► Soft Computing
► How Soft Computing ?
► Hard Computing vs. Soft Computing
► Hybrid Computing
2
 CONCEPT OF COMPUTATION

Computing
y=f(x)
Input/Antecedent Output/Consequent

Control action
y=f(x) is called a mapping function
f= formal method or an algorithm to
3
solve a problem
IMPORTANT CHARACTERISTICS OF COMPUTING

► Should provide precise solution.

► Control action should be unambiguous and accurate.
► Suitable for problem, which is easy to model
mathematically.

4
 HARD COMPUTING

► Hard computing includes

 Guarantee of precise result
 Control action is unambiguous
 Controlaction is formally defined (i.e. with mathematical
model or algorithm)

5
 EXAMPLES OF HARD COMPUTING

► Solving numerical problems (e.g. roots of polynomials,

integration, etc.).
► Searching and sorting techniques.
► Solving computational geometry problems (e.g. shortest
tour in a graph, finding closest pair of points in given a set
of points, etc.).

6
7
 ROLE OF SOFT COMPUTING
• The term “soft computing” was introduced by Professor
Lotfi Zadeh with the objective of exploiting the
tolerance for imprecision, uncertainty and partial truth
to achieve tractability, robustness, low solution cost and
better rapport with reality.
• Its principal constituents are fuzzy logic,
neurocomputing, and probabilistic reasoning. Soft
computing is likely to play an increasingly important
role in many application areas, including software
engineering. The role model for soft computing the
8

human mind.”
 CHARACTERISTICS OF SOFT COMPUTING

• It does not require any mathematical modeling for problem

solving.
• It may not yield the precise solution.
• Algorithms are adaptive (i.e. it can adjust to the change of
dynamic environment).
• Use some biological inspired methodologies such as genetics,
evolution, ant’s behaviors, particles swarming, human nervous
system, etc.
9
 SOFT COMPUTING

cat
Animal? cat
cut
Neural character
recognition

knowledge

Animal character recognition

DIFFERENCE BETWEEN HARD AND SOFT
COMPUTING
Hard Computing Soft Computing

► It requires a precisely stated analytical ► It is tolerant of imprecision uncertainty,

model and often a set of computation partial truth and approximation.
time.
It is based on fuzzy logic, neural
► It is based on binary logic, crisp system, networks, probabilistic reasoning etc.
neural analysis and crisp software.
► It has the characteristics of approximation
► It has the characteristics of precision and and disposition ability.
categoricity
► Stochastic
► Deterministic
► Can deal with ambiguous and noisy data
► Requires exact input data
► Parallel computations
► Strictly sequential
► Yields approximation answer
► Produces precise answer

11
Current Applications using Soft Computing

• Handwriting recognition
• Automotive systems and manufacturing
• Image processing and data compression
• Architecture
• Decision-support systems
• Data Mining
• Power systems
• Control Systems

12
 ARTIFICIAL NEURAL NETWORK (ANN)
Neural networks are those information processing systems, which are
constructed and implemented to model the human brain. The main objective
of the neural network research is to develop a computational device for
modeling the brain to perform various computational tasks at a faster rate
than the traditional systems.

An artificial neural network (ANN) is an efficient information processing

system which resembles in characteristics with a biological neural network.
ANNs possess large number of highly interconnected processing elements
called nodes or units or neurons, which usually operate in parallel and are
configured in regular architectures.

13
 ARTIFICIAL NEURAL NETWORK (ANN)
Each neuron is connected with the other by a connection link. Each
connection link is associated with weights which contain information about
the input signal. This information is used by the neuron net to solve a
particular problem.

ANNs’ collective behavior is characterized by their ability to learn, recall

and generalize training patterns or data similar to that of a human brain.
They have the capability to model networks of original neurons as found in
the brain.

 Thus, the ANN processing elements are called neurons or artificial neurons.

14
 Advantages of Neural Networks:-

a. Adaptive learning
b. Self-organization
c. Real-time operation
d. Fault tolerance via redundant information coding

15
Architecture of a simple artificial neuron network

16
ARTIFICIAL NEURAL NETWORK (ANN)

x1
X1

Y
y
x2
X2

yin=x1ω1+x2ω2
Y=f(yin)
o/p=function (net i/p calculated)
17
 The Brain vs. Computer

BRAIN COMPUTER

1. 10 billion neurons 1. Faster than neuron (10-9 sec)

2. 60 trillion synapses cf. neuron: 10-3 sec
3. Distributed processing 2.Central processing
4. Parallel processing 3.Arithmetic operation
5. Nonlinear processing (linearity)
4. Sequential processing

18
Biological Neural Networks (BNN)

19
Biological Neural Networks (BNN)

20
Comparison Between Biological Neuron and Artificial Neuron
(Brain vs. Computer)
► Speed: The cycle time of execution in the ANN is of few nanoseconds whereas in the case of
biological neuron it is of a few milliseconds. Hence, the artificial neuron modeled using a computer is
more faster.
► Processing: Basically, the biological neuron can perform massive parallel operations simultaneously.
The artificial neuron can also perform several parallel operations simultaneously, but, in general,
the artificial neuron network process is faster than that of the brain.
Size and complexity: The complexity of the brain is comparatively higher, i.e. the computational
work takes place not only in the brain cell body, but also in axon, synapse, etc. On the other hand,
the size and complexity of an ANN is based on the chosen application and the network designer. The
size and complexity of a biological neuron is more than that of an artificial neuron.
► Storage capacity (memory): The biological neuron stores the information in its interconnections or in
synapse strength but in an artificial neuron it is stored in its contiguous memory locations. In an
artificial neuron, the continuous loading of new information may sometimes overload the memory
locations. As a result, some of the addresses containing older memory locations may be destroyed.
But in case of the brain, new information can be added in the interconnections by adjusting the
strength without destroying the older information. A disadvantage related to brain is that sometimes
its memory may fail to recollect the stored information whereas in an artificial neuron, once the
3 0
information is stored in its memory locations, it can be retrieved. Owi n g to these facts, the
adaptability is more toward an artificial neuron.
Comparison Between Biological Neuron and Artificial Neuron
(Brain vs. Computer)
► Tolerance: The biological neuron possesses fault tolerant capability whereas the artificial neuron has
no fault tolerance.
► Control mechanism: In an artificial neuron modeled using a computer, there is a control unit present
in Central Processing Unit, which can transfer and control precise scalar values from unit to unit, bur
there is no such control unit for monitoring in the brain.

22
Basic Models of Artificial Neural Network
► The models of ANN are specified by the three basic
entities namely:
1. The model’s synaptic interconnections;
2. Thetraining or learning rules adopted for updating and
adjusting the connection weights;
3. Their activation functions

23
Connections
► The arrangement of neurons to form layers and the connection pattern formed within and between
layers is called Network Architecture. There exist five types of neuron connection architectures. They
are:
1. Single-layer feed-forward Network
2. Multilayer feed-forward Network
3. Single node with its own feedback
4. Single-layer recurrent Network
5. Multilayer recurrent Network

24
Single-layer feed-forward Network

25
Multilayer feed-forward Network

26
Single node with own feedback

27
Single-layer recurrent network

37
Multilayer recurrent network

29
Learning
► Learningor training is a process by means of which a
neural network adapts itself to a stimulus by making
proper parameter adjustments; resulting in the
production of desired response.
► Thelearning in an ANN can be generally classified into
three categories as:
1. Supervised Learning
2. Unsupervised Learning
3. Reinforcement Learning
30
1- Supervised Learning :-

31
2-Unsupervised Learning

32
3-Reinforcement Learning

33
1---Identity function:
Activation Functions
► It is a linear function and can be defined as f(x) = x for all x
►The output here remains the same as input. The input layer uses the identity activation function.

2---Binary step function:

► This function can be defined as------->
► where θ represents the threshold value. This function is most widely used in single-layer nets to
convert the net input to an output that is a binary (1 or 0).
3---Bipolar step function:
► It is defined as--------->
► where θ represents the threshold value. This function is also used in single-layer nets to convert the
net input to an output that is bipolar (+1 or -1).
4---Sigmoidal functions:
► The sigmoidal functions are widely used in back-propagation nets.
► Sigmoidal functions are of two types:
a) Binary sigmoid function:
► It is also termed as logistic sigmoid function or unipolar sigmoid function.
It can be defined as 34
► where λ is the steepness parameter.
► The derivative of this function is f’(x) = λ f(x)[1-f(x)]. Here the range of the sigmoid function is 0
to 1.
b) Bipolar sigmoid function:
► This function is defined as
► where λ is the steepness parameter and the sigmoid function range is between -1 and +1.
► The derivative of this function can be

5---Ramp function
The ramp function is defined as

35
Weights :-
► In the architecture of an ANN, each neuron is connected to the other neutrons by means of directed
communication links, and each communication link is associated with weights.
► The weights contain information about the input signal. This information is used by the net to solve a
problem.
► The weight can be represented in terms of matrix. The weight matrix can also be called connection
matrix.
► To form a mathematical notation, it is assumed that there are “n” processing elements in an ANN
and each processing element has exactly “m” adaptive weights. Thus, the weight matrix W is defined
by

36
 For the network shown in Figure 1, calculate the net input to the output neuron.

Solution: The given neural net consists of three input neurons and one output neuron. The inputs and weights are

37
Bias :-
► The bias included in the network has its impact in calculating net
input.
► The bias is included by adding a component x0 = 1 to the input
vector X.
► The bias is considered like another weight, that is, w0j = bj.
► The bias can be of two types:
► Positivebias-The positive bias helps in increasing the net input of
the network.
► Negative bias-The negative bias helps in decreasing the net input
of the network.
38
Bias :-
► Consider a simple network shown in Figure with bias.

39
 Calculate the net input for the network shown in Figure 2 with bias included in the
network.

Solution: The given net consists of two input neurons, a bias and an output neuron. The inputs are

40
Threshold :-
► Threshold is a set value based upon which the final output
of the network may be calculated.
► The threshold value is used in the activation function.
► A comparison is made between the calculated net input and
the threshold to obtain the network output.
► For each and every application, there is a threshold limit.

41
 Obtain the output of the neuron Y for the network shown in Figure 3 using activation functions as:
(i) binary sigmoidal and (ii) bipolar sigmoidal.

Solution: The given network has three input neurons with bias and one output neuron. These form a single-layer
network. The inputs are given as

42
 Obtain the output of the neuron Y for the network shown in Figure 3 using activation functions as:
(i) binary sigmoidal and (ii) bipolar sigmoidal.

Solution:

43
Learning Rate :-
► The learning rate is denoted by “α”.
► Itis used to control the amount of weight adjustment at
each step of training.
► The learning rate, ranging from 0 to 1, determines the rate
of learning at each time step.

44
McCulloch-Pitts Neuron
► The McCulloch-Pitts neuron was the earliest neural network discovered in 1943.
► It is usually called as M-P neuron.
► The M-P neurons are connected by directed weighted paths.
► The activation of a M-P neuron is binary, that is, at any time step the neuron may fire
or may not fire.
► The weights associated with the communication links may be excitatory (weight is
positive) or inhibitory (weight is negative).
► The threshold plays a major role in M-P neuron. There is a fixed threshold for each
neuron, and if the net input to the neuron is greater than the threshold then the
neuron fires.
► Also, it should be noted that any nonzero inhibitory input would prevent the neuron
from firing.
► The M-P neurons are most widely used in the case of logic functions. 45
 McCulloch-Pitts Neuron
Architecture
► A simple M-P neuron is shown in the Figure.
► The M-P neuron has both excitatory and inhibitory connections.
► It is excitatory with weight (w > 0) or inhibitory with weight –p (p < 0).
► Since the firing of the output neuron is based upon the threshold, the
activation function here is defined as

► The M-P neuron has no particular training algorithm.

► An analysis has to be performed to determine the values of the weights
and the threshold.
► Here the weights of the neuron are set along with the threshold to make
the neuron perform a simple logic function.
► The M-P neurons are used as building blocks on which we can model any
function or phenomenon, which can be represented as a logic function. 55
Linear separability
► An ANN does not give an exact solution for a nonlinear problem.
However it provides possible approximate solutions to nonlinear
problems.
► Linear separability is the concept wherein the separation of the input
space into regions is based on whether the network response is
positive or negative.
► A decision line is drawn to separate positive or negative responses.
► The decision line may also be called as decision-making line or
decision-support line or linear-separable line.
► Generally, the net input calculated to the output unit is given as

47
Hebb Network
► Donald Hebb stated in 1949 that in the brain, the learning is performed by the change in the
synaptic gap.
► Hebb explained it: “When an axon of cell A is near enough to excite cell B, and repeatedly or
permanently takes place in firing it, some growth process or metabolic change takes place in
one or both the cells such that A’s efficiency, as one of the cells firing B, is increased.”
► According to the Hebb rule, the weight vector is found to increase proportionately to the
product of the input and the learning signal. Here the learning signal is equal to the neuron’s
output.
► In Hebb learning, if two interconnected neurons are ‘on’ simultaneously then the weights
associated with these neurons can be increased by the modification made in their synaptic gap
(strength).
► The Hebb rule is more suited for bipolar data than binary data.
► The weight update in Hebb rule is given by

48
Flowchart of Hebb
training algorithm

49
Question : Design a Hebb net to implement logical AND function (use bipolar inputs and targets)
Solution:

AND function is very simple and mostly known to everyone where the output is 1/SET/ON if both the
inputs are 1/SET/ON. But in the above example, we have used ‘-1' instead of ‘0’ this is because the Hebb
network uses bipolar data and not binary data because the product item in the above equations would give
the output as 0 which leads to a wrong calculation.
Starting with setp1 which is inializing the weights and bias to ‘0’, so we get w1=w2=b=0
A) First input [x1,x2,b]=[1,1,1] and target y = 1. Now using the initial weights as old weight and applying the
Hebb rule(ith value of w(new) = ith value of w(old) + (ith value of x * y)) as follow;
w1(new) = w1(old) + (x1*y) = 0+1 * 1 = 1
w2(new) = w2(old) + (x2*y) = 0+1 * 1 = 1
b(new) = b(old) + y = 0+1 =1
Now the above final weights act as the initial weight when the second input pattern is
presented. And remember that weight change here is;
Δith value of w = ith value of x * y
hence weight changes relating to the first input are;
Δw1= x1y = 1*1=1
Δw2 = x2y = 1*1=1
Δb = y = 1
similarly, using the same process for second, third and fourth row we get a new table as
follows; We got our first output and now we start with the second inputs from the
table(2nd row)