Unit 4 CCS 364 Soft Computing

UNIT 4 NEURO FUZZY MODELLING

ANFIS architecture – hybrid learning- ANFIS as universal approximator - Coactive Neuro fuzzy
modeling – Framework – Neuron functions for adaptive networks – Neuro fuzzy spectrum -
Analysis of Adaptive Learning Capability

4.1 Introduction Hybrid System

 In hybrid system, more than one technology is used. Integrated architectures for machine
learning have been shown to provide performance improvements over single representation
architectures.
 The combination of knowledge based systems, neural networks and evolutionary computation
forms the core of an emerging approach to building hybrid intelligent systems capable of
reasoning and learning in an uncertain and imprecise environment.
 In recent years multiple module integrated machine learning systems have been developed to
overcome the limitations inherent in single component systems. Integrations of neural
networks, fuzzy logic and global optimization algorithms have received considerable attention
but increasing attention is being paid to integrations with case based reasoning and rule
induction.
Classification of hybrid systems :
1.Sequential hybrid systems 2. Auxiliary hybrid systems
3. Embedded hybrid systems
 Sequential hybrid system uses pipeline concept. The output of one technology is used as input
of other technology.
 Auxiliary hybrid system works like subroutine. First technology process or manipulate data
required by it and second technology processes the information provided by the first technology
and return it to first technology.
 Two technologies is integrated such way that they appear intertwined. Embedded system is
hybrid of hardware and software. Neuro-Fuzzy system is an example of embedded hybrid
system

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 1
Unit 4 CCS 364 Soft Computing

4.2 Difference between Neural Processing and Fuzzy Processing

Neural processing Fuzzy processing

It tries to incorporate human Fuzzy logic allows making definite
thinking process to solve decisions based on imprecise or
problems without mathematically ambiguous data
modeling.
Learning is start from scratch. Priori knowledge is required.
Learning algorithm is used Learning algorithm is not used.
Behavior is like black box Simple to implement
Neural networks are good at Fuzzy logic can reason with imprecise
recognizing patterns, they are not information, are good at explaining their
good at explaining how they reach decisions but they cannot automatically
their decisions. acquire the rules they use to make those
decisions

4.2.1 How does a neuro-fuzzy system learn?

 A neuro-fuzzy system is essentially a multi-layer neural network, and thus it can apply
standard learning algorithms developed for neural networks, including the back-
propagation algorithm.
 When a training input-output example is presented to the system, the back-propagation
algorithm computes the system output and compares it with the desired output of the
training example.
 The error is propagated backwards through the network from the output layer to the input
layer. The neuron activation functions are modified as the error is propagated.
 To determine the necessary modifications, the back-propagation algorithm differentiates
the activation functions of the neurons.
 A neuro-fuzzy system is a fuzzy system that uses a learning algorithm derived from or
inspired by neural network theory to determine its parameters by processing data
samples.

4.2.2 Characteristics of Neuro-fuzzy Hybrid Systems

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 2
Unit 4 CCS 364 Soft Computing

1. A neuro-fuzzy system based on an underlying fuzzy system is trained by means of a data-

driven learning method derived from neural network theory.
2. It can be represented as a set of fuzzy rules at any time of the learning process, i.e.,
before, during and after.
3. System may be initialized with or without prior knowledge in terms of fuzzy rules.
4. The learning procedure is constrained to ensure the semantic properties of the underlying
fuzzy system.
5. A neuro-fuzzy system approximates n-dimensional unknown function which is partly
represented by training examples.
4.2.3 Types of Neuro-Fuzzy Systems

1. Cooperative Neuro-Fuzzy System :

 In the cooperative systems there is a pre-processing phase where the neural networks
mechanisms of learning determine some sub-blocks of the fuzzy system. After the fuzzy
sub-blocks are calculated the neural network learning methods are taken away, executing
only the fuzzy system. Fig. 5.2.2 shows cooperative neuro fuzzy system.
 The neural networks are only used in an initial phase.

Fig. 5.2.2
❒ 2. Concurrent Neuro-Fuzzy System :
 In the concurrent systems the neural network and the fuzzy system work continuously together. In
general, the neural networks pre-process the inputs of the fuzzy system. Fig. 5.2.3 shows
concurrent neuro fuzzy system.

 Fig. 5.2.3 : Concurrent Neuro-Fuzzy System

 A concurrent system is not a neuro -fuzzy system in the strict sense, because the neural network
works together with the fuzzy system. This means that the inputs enter in the fuzzy system are pre-
processed and then the neural network processes the outputs of the concurrent system or in the
reverse way.
Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 3
Unit 4 CCS 364 Soft Computing

4.3 ANFIS

ANFIS (Adaptive Neuro-Fuzzy Inference System) is a hybrid intelligent system that combines
the learning capabilities of neural networks with the human-like reasoning style of fuzzy logic.
It’s particularly useful for modeling complex nonlinear functions and is widely used in control
systems, time series prediction, and pattern recognition.

ANFIS Architecture Overview

ANFIS is based on the Sugeno-type fuzzy inference system and typically consists of five
layers, each with specific functions:

1. Input Layer (Layer 1: Fuzzification)

Nodes: Each node represents a fuzzy membership function (MF) like Gaussian, triangular, or
trapezoidal.
Function: Maps the input values to fuzzy membership grades.
OutputExample:
If input is x, and MF is A1, output = μA1(x) — membership value of x in A1.

2. Rule Layer (Layer 2: Rule Firing Strength)

Nodes: Each node corresponds to a fuzzy rule.
Function: Computes the firing strength of each rule by taking the product (AND) of incoming
membership grades.
OutputExample:
wi = μA1(x) × μB1(y) for Rule i.

3. Normalization Layer (Layer 3)

Nodes: Each node calculates the normalized firing strength of a rule.
Function:

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 4
Unit 4 CCS 364 Soft Computing

wiˉ= wi / ∑ jwj
Ensures the rule strengths sum up to 1.

4. Defuzzification Layer (Layer 4: Consequent Layer)

Nodes: Each node implements the Sugeno function:
wiˉ*fi = wiˉ*(pix+qiy+ri)\
Parameters pi,qi,rip_i, q_i, r_ipi,qi,ri are learned during training.

5. Output Layer (Layer 5: Aggregation)

Node: Single node.
Function: Sums all the outputs from the previous layer to produce the final output:
Output=∑iwiˉfi\text
Learning in ANFIS
Hybrid Learning Algorithm: Combines Least Squares Estimation (LSE) for linear
parameters and Gradient Descent for nonlinear ones (membership functions).
It adjusts both the membership function parameters and the consequent parameters to
minimize error.
Application Areas
Time series prediction
Function approximation
System identification
Control systems (e.g., HVAC, robotics)
Example:
Input x ─┐
├──► [Layer 1: Fuzzification] → Apply fuzzy membership functions
Input y ─┘
↓
[Layer 2: Rule Layer] → Evaluate fuzzy rules (e.g., IF x is A1 AND y is B1 THEN...)
↓
[Layer 3: Normalize] → Normalize rule strengths

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 5
Unit 4 CCS 364 Soft Computing

↓
[Layer 4: Consequent Layer] → Apply linear functions like f = p*x + q*y + r
↓
[Layer 5: Output Layer] → Compute weighted sum of outputs

Let’s model this rule-based system:

 Rule 1: IF x is Low AND y is Low, THEN f1 = 1*x + 2*y + 0

 Rule 2: IF x is High AND y is High, THEN f2 = 3*x + 4*y + 1

Let’s say we input:

x = 2, y = 3

Steps:

1. Fuzzification: Determine how "Low" or "High" each input is using membership functions.

 µLow(x=2) = 0.8
 µHigh(x=2) = 0.2
 µLow(y=3) = 0.7
 µHigh(y=3) = 0.3
2. Rule Firing Strengths:

 w1 = µLow(x) * µLow(y) = 0.8 * 0.7 = 0.56

 w2 = µHigh(x) * µHigh(y) = 0.2 * 0.3 = 0.06

3. Normalize:

 w1' = 0.56 / (0.56 + 0.06) ≈ 0.903

 w2' = 0.06 / (0.56 + 0.06) ≈ 0.097

4. Consequent:

 f1 = 1*2 + 2*3 + 0 = 8
 f2 = 3*2 + 4*3 + 1 = 19

5. Final Output:
Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 6
Unit 4 CCS 364 Soft Computing

 Output = w1'*f1 + w2'*f2 ≈ 0.903*8 + 0.097*19 ≈ 9.07

An adaptive neuro-fuzzy inference system or adaptive network-based fuzzy inference

system (ANFIS) is a kind of artificial neural network that is based on Takagi–Sugeno
fuzzy inference system. The technique was developed in the early 1990s. Since it integrates both
neural networks and fuzzy logic principles, it has potential to capture the benefits of both in a
single framework. Its inference system corresponds to a set of fuzzy IF–THEN rules that have
learning capability to approximate nonlinear functions.

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 7
Unit 4 CCS 364 Soft Computing

For simplicity, we assume that the fuzzy inference sytem under consideration has two inputs x
and y and one output z. For a first-orderTakagi-Sugeno fuzzy model, a common rule set with two
fuzzy if-then rules is the following:
■ Rule 1: If x is A1 and y is B1, then f1=p1x+q1y+r1;
■ Rule 2: If x is A2 and y is B2, then f2=p2x+q2y+r2;

Figure (a) A two inputs first order Takagi-Sugeno fuzzy model with two rules; (b) The
equivalent ANFIS architecture.
Figure (a) illustrates the reasoning mechanism for this Takagi-Sugeno model; where nodes of the
same layer have similar functions. (Here we denote the output of the ith node in layer l as Ol,i )
Layer 1 Every node i in this layer is an adaptive node with a node function

where x (or y) is the input to node i and Ai (or Bi-2) is a linguistic label (such as "small" or

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 8
Unit 4 CCS 364 Soft Computing

"large") associated with this node. In other words, O1,i is the membership grade of a fuzzy set
A ( =A1 , A2 , B1 or B2 ) and it specifies the degree to which the given input x (or y) satifies the
quantifier A. Here the membership function for A can be any appropriate parameterized
membership function introduced inhere, such as the generalized bell function:

where {ai, bi, ci} is the parameter set. As the values of these parameters change, the bell-shaped
function varies accordingly, thus exhibiting various forms of membership function for fuzzy
set A. Parameters in this layer are referred to as premise parameters.
Layer 2 Every node in this layer is a fixed node labeled , whose output is the product of all the
incoming signals:

Each node output represents the firing strength of a rule. In general, any other T-norm operators
that perform fuzzy AND can be used as the node function in this layer.
Layer 3 Every node in this layer is a fixed node labeled N. The ith node calculates the ratio of
the ith rule's firing strength to the sum of all rules' firing strenghts:

For convenience, outputs of this layer are called normalized firing strengths.
Layer 4 Every node i in this layer is an adaptive node with a node function:

where wi is a normalized firing strength from layer 3 and {pi, qi, ri} is the parameter set of this
node. Parameters in this layer are referred to as consequent parameters.
Layer 5 The single node in this layer is a fixed node labeled ∑, which computes the overall
output as the summation of all incoming singals:
Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 9
Unit 4 CCS 364 Soft Computing

Hybrid systems:
 A Hybrid system is an intelligent system that is framed by combining at least two intelligent
technologies like Fuzzy Logic, Neural networks, Genetic algorithms, reinforcement
learning, etc.
 The combination of different techniques in one computational model makes these systems
possess an extended range of capabilities.
 These systems are capable of reasoning and learning in an uncertain and imprecise
environment.
 These systems can provide human-like expertise like domain knowledge, adaptation in
noisy environments, etc.

Types of Hybrid Systems:

 Neuro-Fuzzy Hybrid systems

 Neuro Genetic Hybrid systems
 Fuzzy Genetic Hybrid systems

(A) Neuro-Fuzzy Hybrid systems:

 The Neuro-fuzzy system is based on fuzzy system which is trained on the basis of the
working of neural network theory.
 The learning process operates only on the local information and causes only local changes
in the underlying fuzzy system.
 A neuro-fuzzy system can be seen as a 3-layer feedforward neural network.
 The first layer represents input variables, the middle (hidden) layer represents fuzzy rules
and the third layer represents output variables.
 Fuzzy sets are encoded as connection weights within the layers of the network, which
provides functionality in processing and training the model.

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 10
Unit 4 CCS 364 Soft Computing

Working flow:
 In the input layer, each neuron transmits external crisp signals directly to the next layer.
 Each fuzzification neuron receives a crisp input and determines the degree to which the
input belongs to the input fuzzy set.
 The fuzzy rule layer receives neurons that represent fuzzy sets.
 An output neuron combines all inputs using fuzzy operation UNION.
 Each defuzzification neuron represents the single output of the neuro-fuzzy system.
Advantages:
 It can handle numeric, linguistic, logic, etc kind of information.
 It can manage imprecise, partial, vague, or imperfect information.
 It can resolve conflicts by collaboration and aggregation.
 It has self-learning, self-organizing and self-tuning capabilities.
 It can mimic the human decision-making process.
Disadvantages:
 Hard to develop a model from a fuzzy system
 Problems of finding suitable membership values for fuzzy systems
 Neural networks cannot be used if training data is not available.
Applications:
 Student Modelling
 Medical systems
 Traffic control systems
 Forecasting and predictions

(B) Neuro Genetic Hybrid systems:

 A Neuro Genetic hybrid system is a system that combines Neural networks: which
are capable to learn various tasks from examples, classify objects and establish

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 11
Unit 4 CCS 364 Soft Computing

relations between them, and a Genetic algorithm: which serves important search
and optimization techniques.
 Genetic algorithms can be used to improve the performance of Neural Networks and
they can be used to decide the connection weights of the inputs.
 These algorithms can also be used for topology selection and training networks.

Working Flow:
 GA repeatedly modifies a population of individual solutions. GA uses three main types of rules
at each step to create the next generation from the current population:
1. Selection to select the individuals, called parents, that contribute to the population at the
next generation
2. Crossover to combine two parents to form children for the next generation
3. Mutation to apply random changes to individual parents in order to form children
 GA then sends the new child generation to ANN model as a new input parameter.
 Finally, calculating the fitness by the developed ANN model is performed.
Advantages:
Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 12
Unit 4 CCS 364 Soft Computing

 GA is used for topology optimization i.e to select the number of hidden layers, number of
hidden nodes, and interconnection pattern for ANN.
 In GAs, the learning of ANN is formulated as a weight optimization problem, usually using
the inverse mean squared error as a fitness measure.
 Control parameters such as learning rate, momentum rate, tolerance level, etc are also
optimized using GA.
 It can mimic the human decision-making process.
Disadvantages:
 Highly complex system.
 The accuracy of the system is dependent on the initial population.
 Maintenance costs are very high.
Applications:
 Face recognition
 DNA matching
 Animal and human research
 Behavioral system

(C) Fuzzy Genetic Hybrid systems:

 A Fuzzy Genetic Hybrid System is developed to use fuzzy logic-based techniques for
improving and modeling Genetic algorithms and vice-versa.
 Genetic algorithm has proved to be a robust and efficient tool to perform tasks like
generation of the fuzzy rule base, generation of membership function, etc.
Three approaches that can be used to develop such a system are:
1. Michigan Approach
2. Pittsburgh Approach
3. IRL Approach

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 13
Unit 4 CCS 364 Soft Computing

Working Flow:
 Start with an initial population of solutions that represent the first generation.
 Feed each chromosome from the population into the Fuzzy logic controller and compute
performance index.
 Create a new generation using evolution operators till some condition is met.
Advantages:
 GAs are used to develop the best set of rules to be used by a fuzzy inference engine
 GAs are used to optimize the choice of membership functions.
 A Fuzzy GA is a directed random search over all discrete fuzzy subsets.
 It can mimic the human decision-making process.
Disadvantages:
 Interpretation of results is difficult.
 Difficult to build membership values and rules.
 Takes lots of time to converge.
Applications:
 Mechanical Engineering
 Electrical Engine
 Artificial Intelligence
 Economics
Hybrid Learning Algorithm

i) The ANFIS can be trained by a hybrid learning algorithm.

ii) In the forward pass the algorithm uses least-squares method to identify the consequent
parameters on the layer 4.
Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 14
Unit 4 CCS 364 Soft Computing

iii) In the backward pass the errors are propagated backward and the premise parameters are
updated by gradient descent.

Basic Learning Rule Definitions

Suppose that an adptive network has L layers and the kth layer has #( k) nodes.

We can denote the node in the ith position of the kth layer by (k, i ). The node function is
denoted by Oi k .

Since the node output depends on its incoming signals and its parameter set (a, b, c ), we have O
k
i= O k i ( Oi k − 1 , . . . , O#( k −1) k − 1, a, b, c )

Notice that O k i is used as both node output and node function.

Error Measure

Assume that a training data set has P entries. The error measure for the pth entry can be defined
as the sum of the squared error
#(L)
Ep = I, (Tm,p−OL )2
m=l m,p

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 15
Unit 4 CCS 364 Soft Computing

Tm,p is the mth component of the pth target.

L
O m,p

is the mth component the actual output vector. The overall error is

E=I,p Ep
p=l

ANFIS is a Universal Aproximator

A universal approximator is a system that can approximate any continuous function to an

arbitrary degree of accuracy, given sufficient parameters (like neurons, fuzzy rules, or layers) and
proper training.

Why is ANFIS a universal approximator?

ANFIS combines:

 Neural networks (which are already universal approximators),

 and fuzzy logic (which can model complex, nonlinear systems using fuzzy rules).

This combination enables ANFIS to learn complex input–output mappings, making it capable of
approximating any continuous function on a compact set — which satisfies the conditions for being a
universal approximator.

Approximate the function:

f(x)= sin(x), for x ∈ [0,2π]

Steps in ANFIS:

1. Generate Data:

 Input: x values from 0 to 2π

 Output: y = sin(x)

2. Initialize Fuzzy Inference System (FIS):

 Choose membership functions (e.g., Gaussian or triangular).

 Define fuzzy rules (either manually or from data using clustering).

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 16
Unit 4 CCS 364 Soft Computing

3. Train Using Hybrid Learning:

 Use gradient descent and least squares to adjust parameters.

4. Test Model:

 Check how well ANFIS predicts sine values.

When the number of rules is not restricted, a zero-order Sugeno model has unlimited
approximation power for matching well any nonlinear function arbitrarily on a compact set. This can
be proved using the Stone-Weierstrass theorem. Let domain D be a compact space of N dimensions,
and let F be a set of continuous real-valued functions on D satisfying the following criteria:

Stone-Weierstrauss theorem – I

Indentity function: The constant f ( x) = 1 is in F. Separability: For any two points x1 ≠ x 2 in D,

there is an f in F such that f ( x1 ) ≠f ( x 2 ).

Algebraic closure: If f and g are any two functions in F, then fg and af + bg are in F for any two
real numbers a and b.

Indentity Function
Indentity function: The constant f ( x) = 1 is in F. The first hypothesis requires that our fuzzy
inference system be able to compute the identity function f ( x) = 1. An obvious solution is to set
the consequence part of each rule equal to one.

Separability: For any two points x1 # x2 in D, there is an f in F such that f(x1) # f(x2). The
second hypothesis requires that our fuzzy inference system be able to compute functions that
have different values for different points. This is achievable by any fuzzy inference system with
appropriate parameters.

Algebraic closure addition: If f and g are any two functions in F, then af + bg are in F for any
two real numbers a and b. • Suppose that we have two fuzzy inference systems S and Sˆ; each of
them has two rules. • The final output of each system is specified as

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 17
Unit 4 CCS 364 Soft Computing

w. f1 + ws2. f2
S: Z = w1 + w2
4.4 CANFIS (coactive neuro fuzzy inference system)

 CANFIS stands for CoActive Neuro-Fuzzy Inference System.

 It is a hybrid intelligent system that combines the learning ability of neural networks with
the reasoning capability of fuzzy logic systems.
 CANFIS is particularly useful in tasks involving function approximation, control, and
pattern recognition.

What is CANFIS?

 "Coactive" refers to the integration of neural networks and fuzzy logic in a cooperative
manner.
 Neuro: The system uses a neural network to learn from data.
 Fuzzy Inference System: It models imprecise or uncertain information using fuzzy logic
rules.

How CANFIS Works

1. Input Layer: Receives crisp inputs.

2. Fuzzification Layer: Uses fuzzy membership functions (e.g., Gaussian, triangular) to convert
crisp inputs into fuzzy values.
3. Rule Layer: Applies fuzzy rules (e.g., IF-THEN rules) that relate inputs to outputs.
4. Neural Network Component: Adjusts the parameters of membership functions and rule
weights using learning algorithms (like backpropagation).
5. Defuzzification Layer: Converts the fuzzy outputs back into crisp values.

Key Features

 Adaptive Learning: Like neural networks, CANFIS can learn from data to improve its
performance.
 Interpretable Logic: The fuzzy logic part allows the system to reason in human-like terms
(e.g., “high”, “low”).
 Better Generalization: Combines rule-based decision making with learning, which often
leads to better generalization on unseen data.

Applications

 Time series prediction (e.g., stock market, weather)

 Pattern recognition
 Control systems (e.g., robotics, industrial automation)
 Medical diagnosis
 Adaptive signal processing

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 18
Unit 4 CCS 364 Soft Computing

CANFIS vs ANFIS
Feature CANFIS ANFIS (Adaptive Neuro-Fuzzy Inference System)
Input to FIS Fuzzified input Direct crisp input
Learning Neural network modifies MF Hybrid learning (least squares + backprop)
Architecture NN + Fuzzy inference Tightly integrated fuzzy and learning modules
Flexibility More adaptable to input types More structured but sometimes less flexible

CANFIS has extended basic ideas of its predecessor ANFIS (Adaptive Network based Fuzzy
Inference System).In this ANFIS concept has been extended to any number of input-/output pairs
.In addition, CANFIS yields advantages from non linear fuzzy rules. This CANFIS realizes the
sugeno –type ( or TSK)fuzzy inferencing accomplishing fuzzy ifthen rules such as ,If X is A1
and Y is B1, Then C1=p1X+q1Y+r1.

FRAMEWORK
Toward Multiple Inputs/Outputs Systems
CANFIS has extended the notion of a single-output system, ANFIS, to produce multiple
outputs. One way to get multiple outputs is to place as many ANFIS models side by side
as there are required outputs. In this MANFIS (multiple ANFIS) model, no modifiable
parameters are shared by
the juxtaposed ANFIS models. That is, each ANFIS has an independent set of fuzzy
rules, which makes it difficult to realize possible certain correlations between outputs. An
additional concern resides in the number of adjustable parameters, which drastically
increases as outputs increase.
Another way of generating multiple outputs is to maintain the same antecedents of fuzzy
rules among multiple ANFIS models.

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 19
Unit 4 CCS 364 Soft Computing

Architectural Comparisons

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 20
Unit 4 CCS 364 Soft Computing

4.5 Neuron functions for adaptive networks

 In adaptive networks, neurons serve as the basic computational units and are designed to
adjust their parameters (like weights and biases) in response to input data during training.
 These neurons are a fundamental part of models like Artificial Neural Networks (ANNs),
and are especially important in adaptive learning systems, such as Adaptive Neural
Networks and Adaptive Neuro-Fuzzy Inference Systems (ANFIS).

Core Neuron Functions in Adaptive Networks

1. InputFunction
Neurons receive multiple inputs (either raw data or from other neurons) and compute a
weighted sum:

U = ∑ i = 1 to n wi xi + b

where:

 xi = inputs
 wi = corresponding weights
 b = bias
 u = net input to the neuron

2. ActivationFunction
This function transforms the input signal to an output signal, introducing non-linearity:

 Sigmoid
 Tanh: \u
 ReLU:
 Linear:

3. OutputFunction

 Often just the result of the activation function, but in some networks (e.g., in ANFIS),
the output may be further processed (e.g., weighted outputs or defuzzification).

4. Adaptation(Learning)Function
This is what makes the network "adaptive". Neurons update their internal parameters
(weights and biases) based on an error signal, using a learning algorithm such as:

 Backpropagation (gradient descent) for traditional ANNs

 Least squares estimation in ANFIS
 Hebbian learning, reinforcement learning, etc.
Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 21
 Unit 4 CCS 364 Soft Computing

5. ErrorCalculationFunction
The error is usually computed as the difference between the desired output ddd and the actual
output y:

e=d−y

This is used in the loss function (e.g., Mean Squared Error) to guide learning.

Neuron Functions in Specific Adaptive Networks

 In ANFIS (Adaptive Neuro-Fuzzy Inference Systems):

 Neurons in different layers serve specific roles like fuzzy membership evaluation, rule
strength computation, normalization, and output calculation.

 In Self-Organizing Maps (SOMs):

 Neurons compete to represent input data, adapting their weights via unsupervised
learning.

 In Recurrent Neural Networks (RNNs):

 Neurons have feedback loops that let them retain memory and adapt based on
temporal patterns.

4.6 Neuro-Fuzzy Spectrum

The neuro-fuzzy spectrum refers to the range of hybrid intelligent systems that combine neural
networks and fuzzy logic, with different approaches occupying various points along a conceptual
spectrum between pure neural networks and pure fuzzy systems.

Key Concepts

1. Neural Networks: Computational models inspired by biological neural networks, capable of learning
complex patterns from data.

2. Fuzzy Logic: A form of many-valued logic that handles approximate reasoning by allowing variables
to have truth values between 0 and 1.

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 22
 Unit 4 CCS 364 Soft Computing

Points on the Neuro-Fuzzy Spectrum

1. Pure Neural Networks

 Traditional ANNs (MLPs, RNNs, CNNs)

 Black-box models with strong learning capabilities

2. Neural Networks with Fuzzy Elements

 Fuzzy activation functions

 Fuzzy neurons (processing fuzzy inputs)

 Neural networks for fuzzy rule optimization

3. Cooperative Neuro-Fuzzy Systems

 Separate neural and fuzzy components that work together

 Neural networks pre-process inputs for fuzzy systems

 Fuzzy systems post-process neural network outputs

4. Concurrent Neuro-Fuzzy Models

 ANFIS (Adaptive Neuro-Fuzzy Inference Systems)

 Fuzzy neural networks with interpretable rules

 Simultaneous learning and fuzzy reasoning

5. Pure Fuzzy Systems

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 23
 Unit 4 CCS 364 Soft Computing

 Traditional fuzzy inference systems

 Rule-based with human-defined membership functions

Applications Across the Spectrum

 Left side (Neural-dominated): Image recognition, complex pattern detection

 Middle (Balanced): Predictive maintenance, financial forecasting

 Right side (Fuzzy-dominated): Control systems, decision support with interpretable rules

Advantages of Neuro-Fuzzy Systems

1. Combines learning capability (from neural networks) with interpretability (from fuzzy systems)

2. Can handle both numerical data and linguistic knowledge

3. More robust to noise and uncertainty than pure approaches

4. Allows for gradual transition between data-driven and knowledge-driven approaches

The optimal point on the neuro-fuzzy spectrum depends on the specific application
requirements regarding accuracy, interpretability, and available data/knowledge.

4.7 Analysis of Adaptive Learning Capability

Definition
Adaptive learning capability refers to an organization's or system's ability to adjust its
strategies, processes, and behaviors in response to changing internal and external environments. It
involves continuous learning, flexibility, and the capacity to evolve based on feedback and new
information.

Key Components

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 24
Unit 4 CCS 364 Soft Computing

 Sensing Mechanisms: Ability to detect changes in the environment (market trends,

technological advancements, competitor moves).
 Information Processing: Efficient analysis of data to derive actionable insights.

 Experimentation & Innovation: Willingness to test new approaches and learn from failures.

 Agility in Decision-Making: Quick adaptation of policies, structures, and operations.

 Knowledge Management: Effective storage, sharing, and application of learned experiences.

Importance
 Competitive Advantage: Organizations with strong adaptive learning capabilities outperform
rigid competitors.
 Resilience: Better equipped to handle disruptions (e.g., economic shifts, pandemics).

 Innovation: Encourages continuous improvement and creative problem-solving.

 Employee & Customer Satisfaction: Adaptability leads to responsive and personalized

experiences.

Measurement & Indicators

 Learning Speed: How quickly an organization identifies and implements changes.

 Feedback Integration: Use of customer/employee feedback to refine processes.

 Failure Recovery: Ability to pivot after setbacks.

 Adoption of New Technologies: Readiness to leverage AI, automation, etc.

Challenges

 Resistance to Change: Cultural or structural inertia hindering adaptation.

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 25
Unit 4 CCS 364 Soft Computing

 Information Overload: Difficulty in filtering relevant data for decision-making.

 Short-Term Focus: Prioritizing immediate gains over long-term learning.

Examples

 Netflix: Shifted from DVD rentals to streaming, then to content creation.

 Amazon: Continuously evolves its logistics, AI, and customer service strategies.

 Tesla: Over-the-air software updates improve vehicle performance post-purchase.

Enhancing Adaptive Learning

 Promote a Learning Culture: Encourage curiosity and knowledge-sharing.

 Leverage Technology: Use AI, big data analytics, and digital platforms for real-time insights.

 Decentralize Decision-Making: Empower teams to respond swiftly to changes.

 Invest in Training: Upskill employees to handle new challenges.

Future Trends

 AI-Driven Adaptation: Machine learning models predicting and automating adjustments.

 Cross-Industry Learning: Borrowing adaptive strategies from unrelated sectors.

 Sustainability Integration: Adapting to climate and regulatory changes.

Dr.V.KalaivazhiB.E.,M.Tech.,Ph.D
Page 26