Hybrid Systems

Hybrid Systems
• Hybrid Soft Computing Systems leverage the strengths of different soft
computing methodologies to create powerful and flexible systems capable of
solving complex real-world problems. They are particularly useful in situations
where the data is uncertain, the relationships are nonlinear, and there is a
need for adaptability and learning.

• The combined use of fuzzy systems, neural networks, and genetic algorithms
has led to the development of intelligent systems that can learn from data,
adapt to changes, and make decisions in the presence of uncertainty.
 Necessary of Hybrid Systems
• Handling Uncertainty and Imprecision: Real-world data is often noisy, imprecise, and
uncertain. Fuzzy Logic is well-suited for handling such data, while ANNs are good at
learning from examples and capturing nonlinear relationships. Combining these two
can lead to systems that can learn from imprecise data and make intelligent decisions.

• Optimization and Learning: Genetic Algorithms are optimization techniques inspired

by the process of natural selection. When combined with ANNs, GAs can be used to
optimize the weights and architecture of the neural networks, leading to better
performance.
 Necessary of Hybrid Systems…
• Improving Interpretability and Adaptability: Fuzzy systems provide a way to
incorporate expert knowledge and are interpretable, but they might not adapt well to
changes in the environment or data. Neural networks can learn and adapt, but are
often considered "black boxes". Combining these two can lead to systems that are
both adaptable and interpretable.
 Necessary of Hybrid Systems…
• Combined Use of Fuzzy and ANN (Neuro-Fuzzy Systems): Neuro-Fuzzy Systems integrate
fuzzy logic and neural networks. The fuzzy system brings in the ability to reason with linguistic
terms and handle uncertainty, while the neural network brings in learning capabilities.
• Learning Membership Functions: Neural networks can be used to learn the parameters of the
fuzzy membership functions from data.
• Adaptive Systems: The combination leads to systems that can adapt to changes in the
environment and still make decisions in the presence of uncertainty.
• Example: ANFIS (Adaptive Neuro-Fuzzy Inference System) is a popular neuro-fuzzy system that
combines these methodologies to create a system that can learn and adapt.
 Necessary of Hybrid Systems…
• Combined Use of ANN-GA: This combination is often used for optimization problems where
GA is used to optimize the structure and parameters of the neural network.
• Weight Optimization: GA can be used to find the optimal set of weights for a neural network.
• Feature Selection: GA can help in selecting the most relevant features for the neural network,
leading to better performance.
• Model Selection: Selecting the best neural network architecture for a given problem.
Neuro-Fuzzy System
 Neuro-Fuzzy System
• A neuro-fuzzy system is a fuzzy system that uses a learning algorithm derived from or
inspired by neural network theory to determine its parameters (fuzzy sets and fuzzy rules)
by processing data samples.

• It can also be defined as a fuzzy system that determines its parameters by processing data
samples by using a learning algorithm derived from or inspired by neural network theory.
Alternately, it is a hybrid intelligent system that fuses artificial neural networks and fuzzy
logic by combining the learning and connectionist structure of neural networks with
human-like reasoning style of fuzzy systems.

• Modern neuro-fuzzy systems are usually represented as special multilayer feedforward

neural networks like ANFIS, FuNe, Fuzzy RuleNet , GARIC, or NEFCLASS and NEFCON.
 Neuro-Fuzzy System…
• Neuro-fuzzy hybridization is widely termed as Fuzzy Neural Network (FNN) or Neuro-Fuzzy
System (NFS).

• The human-like reasoning style of fuzzy systems is incorporated by NFS (the more popular
term is used henceforth) through the use of fuzzy sets and a linguistic model consisting of a
set of IF–THEN fuzzy rules. NFSs are universal approximators with the ability to solicit
interpretable IF–THEN rules; this is their main strength.

• However, the strength of NFSs involves interpretability versus accuracy, requirements that
are contradictory in fuzzy modeling.
• In the field of fuzzy modeling research, the neuro-fuzzy is divided into two areas:
• Linguistic fuzzy modeling focused on interpretability (mainly the Mamdani model).
• precise fuzzy modeling focused on accuracy [mainly the Takagi–Sugeno–Kang (TSK)
model]
 Neuro-Fuzzy System…
• From the existing literature, it can be noted that neural networks and fuzzy systems have
some things in common.

• If there does not exist any mathematical model of a given problem, then neural networks and
fuzzy systems can be used for solving that problem (e.g., pattern recognition, regression, or
density estimation).

• This is the main reason for the growth of these intelligent computing techniques. Besides
having individual advantages, they do have certain disadvantages that are overcome by
combining both concepts.

• When neural networks are concerned, if one problem is expressed by sufficient number of
observed examples then only it can be used. These observations are used to train the black
box. Though no prior knowledge about the problem is needed, extracting comprehensible
rules from a neural network’s structure is very difficult. In this case fuzzy logic provides the
logic to define the rules.
 Neuro-Fuzzy System…
• From the existing literature, it can be noted that neural networks and fuzzy systems have
some things in common.

• If there does not exist any mathematical model of a given problem, then neural networks and
fuzzy systems can be used for solving that problem (e.g., pattern recognition, regression, or
density estimation).

• This is the main reason for the growth of these intelligent computing techniques. Besides
having individual advantages, they do have certain disadvantages that are overcome by
combining both concepts.

• When neural networks are concerned, if one problem is expressed by sufficient number of
observed examples then only it can be used. These observations are used to train the black
box. Though no prior knowledge about the problem is needed, extracting comprehensible
rules from a neural network’s structure is very difficult. In this case fuzzy logic provides the
logic to define the rules.
 Neuro-Fuzzy System…
A fuzzy system, on the other hand, does not need learning examples as prior knowledge; rather
linguistic rules are required. Moreover, linguistic description of the input and output variables
should be given. If the knowledge is incomplete, wrong or contradictory, then the fuzzy system
must be tuned. This is a time-consuming process.

Above table shows how combining both approaches brings out the advantages, leaving out the
disadvantages.
 Neuro-Fuzzy System…
• The general architecture of neuro-fuzzy hybrid system is as shown in Figure. A fuzzy system-based
NFS is trained by means of a data-driven learning method derived from neural network theory. This
heuristic causes local changes in the fundamental fuzzy system. At any stage of the learning process
– before, during, or after – it can be represented as a set of fuzzy rules. For ensuring the semantic
properties of the underlying fuzzy system, the learning procedure is constrained.
 Neuro-Fuzzy System…
An NFS approximates an n-dimensional unknown function, partly represented by training examples.
Thus fuzzy rules can be interpreted as vague prototypes of the training data. As shown in Figure, an
NFS is given by a three-layer feedforward neural network model. It can also be observed that the
first layer corresponds to the input variables, and the second and third layers correspond to the
fuzzy rules and output variables, respectively.

The fuzzy sets are converted to (fuzzy) connection weights. NFS can also be considered as a system
of fuzzy rules wherein the system can be initialized in the form of fuzzy rules based on the prior
knowledge available. Some researchers use five layers – the fuzzy sets being encoded in the units of
the second and the fourth layer, respectively. It is, however, also possible for these models to be
transformed into three-layer architecture.
 Neuro-Fuzzy System..
NFSs can be classified into the following two systems:
1. Cooperative NFSs.
2. General neuro-fuzzy hybrid systems.
 Cooperative NFS
• In this type of system, both artificial neural network (ANN) and fuzzy
system work independently from each other. The ANN attempts to
learn the parameters from the fuzzy system.
 Cooperative NFS

In this model, the neural network is used to learn fuzzy sets from the training data. Membership functions, which
define how each point in the input space is mapped to a membership value between 0 and 1, are fitted using a
neural network. After the membership functions are determined, they are used offline, meaning that the learned
fuzzy sets are fixed and not updated further during the system operation. The fuzzy system then uses predefined
fuzzy rules that are not learned but are set by the designer. The outcome is a fuzzy system that operates with
fixed fuzzy sets and rules based on the initial training.
 Cooperative NFS

Here, the neural network's role is to determine the fuzzy rules from the training data. Similar to the
previous model, the neural network performs its learning offline. Rule learning typically involves clustering,
which can be done using self-organizing feature maps or fuzzy clustering methods. The fuzzy system is then
initialized with these learned rules, which dictate how the system processes inputs and generates outputs.
Cooperative NFS…
• This neuro-fuzzy model is slightly different as it learns the
parameters of the membership functions online, meaning
that it can update its knowledge in real-time as it operates.
• Initially, the fuzzy rules and membership functions must be
defined, but as data is processed, the system can adapt and
refine these membership functions.
• The system includes an error computing module to
measure the performance of the system and guide its
learning.
• This dynamic approach allows the NFS to continuously
improve and adapt to new data.
Cooperative NFS…
• In this configuration, the neural network
determines the rule weights for all fuzzy rules.
• A rule weight represents the influence or
importance of a rule within the system.
• The weights are then used in combination with the
rule outputs. The output of each rule is multiplied
by its weight to determine its contribution to the
final output.
• This allows for a more nuanced and weighted
approach to applying fuzzy rules, with the neural
network fine-tuning the influence of each rule
based on the training data.
 Cooperative NFS…
• The configurations represent a spectrum of learning where (A) focuses on learning fixed sets, (B)
determines rules from data, (C) allows for continuous online learning of membership functions,
and (D) adjusts the influence of rules through learned weights. These systems are used for tasks
that benefit from fuzzy logic’s ability to handle imprecision and uncertainty, combined with neural
networks' capability to learn from data.

• Let's consider a simple example of a neuro-fuzzy system designed to control the temperature of a
room. The system takes as input the current room temperature and the desired temperature and
outputs the amount of heating or cooling needed.
 Cooperative NFS…
• Figure (A) Learning Fuzzy Sets: In this scenario, a neural network is trained
with data that includes current room temperatures and the corresponding
levels of heating or cooling applied. The goal is to learn fuzzy sets that can
categorize the temperature into fuzzy categories like "Cold," "Warm," or
"Hot."

• Training Data: Current room temperature and required heating/cooling

level.
• Neural Network: Learns membership functions for fuzzy sets "Cold,"
"Warm," "Hot."
• Fuzzy Sets: Defined categories like "Cold," "Warm," "Hot" based on
learned membership functions.
• Fuzzy Systems: Uses predefined rules like "If Cold, then Heat strongly" to
control temperature.
 Cooperative NFS…
• Figure (B) Determining Fuzzy Rules:
• Here, the neural network is employed to learn the fuzzy rules themselves
from the training data, possibly through a process like clustering.
• Training Data: Current room temperature and required heating/cooling level.
• Neural Network: Identifies patterns and clusters in the data.
• Fuzzy Rules: Generated from the patterns, such as "If temperature is slightly
below desired, then apply moderate heating."
• Fuzzy Systems: Initialized with the learned rules and controls the
temperature accordingly.
 Cooperative NFS…
• Figure(C) - Learning Membership Function Parameters Online:
• In this model, the system starts with predefined fuzzy rules and membership
functions, but it adjusts the parameters of the membership functions in real-
time.
• Fuzzy membership function: Initially predefined but adjustable.
• Training Data: Feedback loop with current temperature and system's
performance.
• Error Computing Module: Measures how well the system is maintaining the
desired temperature.
• Fuzzy Systems: Adjusts the parameters of the membership functions to improve
performance, based on the error measurements.
 Cooperative NFS…
• Figure(C) - Learning Membership Function Parameters Online:
• In this model, the system starts with predefined fuzzy rules and membership
functions, but it adjusts the parameters of the membership functions in real-
time.
• Fuzzy membership function: Initially predefined but adjustable.
• Training Data: Feedback loop with current temperature and system's
performance.
• Error Computing Module: Measures how well the system is maintaining the
desired temperature.
• Fuzzy Systems: Adjusts the parameters of the membership functions to improve
performance, based on the error measurements.
 Cooperative NFS…
• Figure (D) - Determining Rule Weights:
• In this case, the system uses a neural network to learn how much weight to
assign to each fuzzy rule, which influences the final decision.
• Rule weights: Learned from how effective each rule is based on training data.
• Training Data: Current room temperature, desired temperature, and system's
performance.
• Neural Network: Adjusts rule weights, increasing or decreasing their influence.
• Fuzzy Systems: Applies the weighted rules to control the temperature, with
some rules having more weightage based on their weights.
 Cooperative NFS…
• This simple temperature control example illustrates how neuro-fuzzy systems
can be configured to learn and adapt in various ways, using neural networks to
learn about fuzzy sets, rules, membership function parameters, and rule weights
to make intelligent decisions based on fuzzy logic.
General Neuro-Fuzzy Hybrid Systems (General NFHS)

• General neuro-fuzzy hybrid systems (NFHS) resemble

neural networks where a fuzzy system is interpreted as a
neural network of special kind. The architecture of general
NFHS gives it an advantage because there is no
communication between fuzzy system and neural network.
Figure illustrates an NFHS.

• In this figure fuzzy sets can be regarded as weights

whereas the input and output variables and the rules are
modeled as neurons. Finally, the neurons of the network
represent the fuzzy knowledge base. Obviously, the major
drawbacks of both underlying systems are thus overcome.
The choice to include or discard neurons can be made in
the learning step. Also, the fuzzy knowledge base is
represented by the neurons of the neural network.
 General Neuro-Fuzzy Hybrid Systems
(General NFHS)…
• General neuro-fuzzy hybrid systems (NFHS) resemble neural networks where a fuzzy system is
interpreted as a neural network of special kind. The architecture of general NFHS gives it an
advantage because there is no communication between fuzzy system and neural network.

• The architecture is advantageous because it eliminates the need for communication between
a separate fuzzy system and a neural network, which simplifies the model and reduces
complexity.

• In the NFHS, fuzzy sets are considered analogous to weights in a neural network. These fuzzy
sets determine the strength or influence of the connections between neurons, similar to how
weights determine the strength of signals in neural networks.

• The input and output variables, along with the fuzzy rules, are modeled as neurons in the
neural network. This means that each neuron in the network can represent a fuzzy rule or a
component of the fuzzy system.
 General Neuro-Fuzzy Hybrid
Systems (General NFHS)…
• The collection of neurons and their interconnections represent the entire fuzzy knowledge
base. The learning process of the neural network allows this knowledge base to adapt and
improve.

• By integrating the fuzzy system within the neural network, NFHS overcomes the limitations
of each system when used independently. Neural networks benefit from the fuzzy logic's
ability to handle imprecision and uncertainty, while fuzzy systems gain from the neural
networks' learning capabilities.

• During the learning phase, decisions are made regarding which neurons (rules) to keep or
discard based on their performance, which is assessed by the error computation.
General Neuro-Fuzzy Hybrid Systems (General NFHS)

• The NFHS consists of several key components:

• Neural Network Module: This module formulates the rule
base using IF-THEN rules. The structure of the neural
network allows it to learn and store knowledge in the form
of fuzzy rules.
• System State: Represents the current status or inputs of
the system under consideration, which could be any
dynamic system that requires control or decision-making.
• Control Output: The output from the NFHS that acts upon
the system under consideration to achieve the desired
state or response.
• Computing Error: This is a feedback mechanism where the
difference between the desired output and the actual
output of the system under consideration is computed.
This error is then fed back into the neural network module
to refine the fuzzy rules and improve the system's
performance over time.
 General Neuro-Fuzzy Hybrid Systems (General NFHS)…
Incorporation of Neural Networks with Fuzzy Logic
• In fuzzy logic, membership functions define how each input maps to a degree of membership between 0
and 1. These functions are associated with linguistic terms that are used in the fuzzy inference rules, such as
"cold," "warm," or "hot" for temperature control systems. The membership functions give fuzzy systems the
ability to handle imprecise information and make decisions in a manner similar to human reasoning.

• The challenge in designing fuzzy systems is that there is no standard, formalized method to determine the
exact shape and parameters of these membership functions. They can take various forms, such as Gaussian
(bell-shaped curves), triangular, or trapezoidal, each characterized by a set of parameters that define their
width, center, and slope.
• The selection and optimization of these membership functions are critical for the performance of a fuzzy
system. Ideally, they should be shaped in a way that allows the fuzzy system to generalize well from the data
it's trained on, meaning it can make accurate inferences about new, unseen data.

• Neural networks can be employed to address this optimization problem. By using learning algorithms, a
neural network can adjust the parameters of the membership functions to better fit the training data. For
instance, in the case of triangular membership functions, a neural network could learn the most suitable
base width and peak position for each linguistic term in the context of the control task at hand.
 General Neuro-Fuzzy Hybrid Systems (General NFHS)…
Incorporation of Neural Networks with Fuzzy Logic…
• Regardless of the membership function shape chosen, having a comprehensive set of training data is
essential. This data must adequately represent the problem space and include examples of the input-output
relationships the fuzzy system is expected to learn.

• By integrating a neural network into the design process, fuzzy systems can benefit from the neural
network's ability to learn from examples, leading to more accurate and robust fuzzy inference systems.
General Neuro-Fuzzy Hybrid Systems (General NFHS)…

• The learning process would typically involve:

• 1. Initialization: Start with an initial guess for the shape and

parameters of the membership functions.
• 2. Training: Use training data (input-output pairs) to train the
neural network. The network will adjust the membership
function parameters to minimize the error between the
system's outputs and the actual desired outputs.
• 3. Evaluation: Assess the performance of the fuzzy system
with the optimized membership functions on validation data.
• 4. Iteration: Repeat the training and evaluation process,
further refining the parameters until the performance on the
validation data stops improving or meets the desired criteria.
 Adaptive Neuro Fuzzy Inference System (ANFIS)
• Adaptive Neuro-Fuzzy Inference System (ANFIS) is a type of artificial neural network that is based on
Takagi-Sugeno fuzzy inference system. It is proposed by Jang (1993)*.

• It combines the learning capabilities of neural networks with the fuzzy logic theory, which allows for
modeling complex, non-linear systems that are difficult to model with traditional techniques. ANFIS
learns to map inputs to outputs through a network structure of nodes and directional links, by adjusting
the membership function parameters of the fuzzy system in a way that the output of the ANFIS model
follows some given target data. This process involves a hybrid learning rule that combines gradient
descent and least-squares methods.

* Jang, J. S. (1993). ANFIS: adaptive-network-based fuzzy inference system. IEEE transactions on systems, man, and
cybernetics, 23(3), 665-685.
 Adaptive Neuro Fuzzy Inference System (ANFIS): Learning
ANFIS learns using a hybrid learning algorithm that combines the least-squares estimator and the
gradient descent method. Initially, it assigns activation functions to each neuron, ensuring sufficient
overlap.

During training, each epoch consists of a forward pass and a backward pass. In the forward pass,
consequent parameters are identified using least-squares estimation. In the backward pass, the error is
propagated back and antecedent parameters are updated via gradient descent, based on the error signal
and the chain rule. This allows ANFIS to learn and tune both the membership functions and rule
parameters for accurate function approximation.
 Adaptive Neuro Fuzzy Inference System (ANFIS): Architecture
1. Input Layer (corresponds to Membership Layer):

Purpose: To receive the input signals and direct them to the next layer.
Function: Each neuron in this layer represents an input variable. Its purpose is to pass the
input values to the next layer without any changes.

2. Fuzzification Layer:
Purpose: To convert crisp input values into fuzzy values based on membership functions.
Function: Neurons in this layer are adaptive and contain membership functions that
determine the degree to which the inputs belong to each of the appropriate fuzzy sets.
Common membership function shapes include Gaussian, trapezoidal, and triangular. The
parameters of these functions can be modified during the learning process to better map
input features to degrees of membership.
 Adaptive Neuro Fuzzy Inference System (ANFIS): Architecture…
3. Rule Layer (sometimes corresponds to Normalisation Layer):

Purpose: To apply fuzzy logic rules which are typically in the form of IF-THEN statements.

Function: Each neuron in this layer represents a fuzzy rule. The output of each neuron is
the result of the application of a fuzzy logic operator (like AND, OR) on the membership
degrees from the previous layer. For instance, if the fuzzy rule is "IF temperature is high
AND humidity is low, THEN increase cooling," the neuron would output the combined
membership degree using an AND operation (often implemented as a min or product
operator).
 Adaptive Neuro Fuzzy Inference System (ANFIS): Architecture…
4. Normalisation Layer:

Purpose: To normalize the rule strengths produced by the previous layer.

Function: Each neuron's output in this layer is the normalized firing strength of a rule. The
firing strength of a rule is divided by the sum of all rules' firing strengths to ensure that the
sum of the output signals of this layer is equal to one. This prepares the data for
defuzzification.
 Adaptive Neuro Fuzzy Inference System (ANFIS): Architecture…
5. Defuzzification Layer:

Purpose: To produce a crisp output from the fuzzy inputs and normalized firing strengths.

Function: The neurons in this layer are also adaptive, and they compute the final output as
a weighted sum of each rule's contribution. This step essentially defuzzifies the fuzzy
quantities into a single crisp output, which is then passed on as the system's response.

6. Summation Layer:

Purpose: It represented by a single summation neuron.

Function: This neuron calculates the sum of outputs of all defuzzification neurons and
produces the overall ANFIS output.
 Adaptive Neuro Fuzzy Inference System (ANFIS): Architecture
Jang’s ANFIS is normally represented by a six-layer feedforward neural network. Figure shows
the ANFIS architecture that corresponds to the first- order Sugeno fuzzy model.
 Adaptive Neuro Fuzzy Inference System (ANFIS): Architecture
Each input is represented by two fuzzy sets, and the output by a first-order polynomial. The
ANFIS implements four rules:
Adaptive Neuro Fuzzy Inference System (ANFIS): Architecture
Adaptive Neuro Fuzzy Inference System (ANFIS): Architecture
Adaptive Neuro Fuzzy Inference System (ANFIS): Architecture
Adaptive Neuro Fuzzy Inference System (ANFIS): Architecture
 Adaptive Neuro Fuzzy Inference System (ANFIS): Architecture

• Thus, the ANFIS shown in Figure is indeed functionally equivalent to a first order Sugeno
fuzzy model.
• However, it is often difficult or even impossible to specify a rule consequent in a polynomial
form. Conveniently, it is not necessary to have any prior knowledge of rule consequent
parameters for an ANFIS to deal with a problem. An ANFIS learns these parameters and
tunes membership functions.
 Concluding Remarks

• A neuro-fuzzy system can be always (i.e. before, during and after learning) interpreted as a system of fuzzy
rules. It is also possible to create the system out of training data from scratch, as it is possible to initialize it
by prior knowledge in form of fuzzy rules.
Remark: Not all neuro-fuzzy models specify learning procedures for fuzzy rule creation.
• The learning procedure of a neuro-fuzzy system takes the characteristics of the underlying fuzzy system into
account. This results in constraints on the possible modifications applicable to the system parameters.
Remark: Not all neuro-fuzzy approaches have this property.
• Membership functions expressing the linguistic terms of the inference rules should be formulated
for building a fuzzy controller. However, in fuzzy systems, no formal approach exists to define these
functions.
• Any shape, such as Gaussian or triangular or bell shaped or trapezoidal, can be considered as a
membership function with an arbitrary set of parameters.
• Thus for fuzzy systems, the optimization of these functions in terms of generalizing the data is very
important; this problem can be solved by using neural networks. Using learning rules, the neural
network must optimize the parameters by fixing a distinct shape of the membership functions; for
example, triangular. But regardless of the shape of the membership functions, training data should
also be available.
Genetic Algorithm and
Neural Network
 Genetic Algorithm and Neural Network
• Neuro-genetic hybrid systems integrate neural networks with genetic algorithms, leveraging
the strength of both. Neural networks are adept at learning and modeling complex patterns
through examples and classifying data, while genetic algorithms excel at optimizing the
neural network's architecture parameters. Despite their efficacy, these hybrid systems face
challenges, such as extensive memory demands for managing genetic data and scalability
issues as network sizes increase. These constraints highlight the need for efficient resource
management and innovation in handling larger neural architectures.
 Genetic Algorithm and Neural Network
• Certain properties of genetic neuro-hybrid systems are as follows:

• In genetic neuro-hybrid systems, the genetic algorithm encodes the parameters of a neural
network into a format akin to chromosomes, effectively representing different potential
solutions. A vast population of these chromosomes is created to represent various possible
parameter configurations of the neural network.

• Genetic Algorithm Neural Network (GANN), is adept at quickly identifying promising

regions in the search space where optimal solutions are likely to be found, showing an
efficiency advantage over traditional search methods.
 Genetic Algorithm Based Back-Propagation Network (BPN)

Limitations:
• BPN do not have the ability to recognize new patterns; they can recognize patterns similar to
those they have learnt.
• They must be sufficiently trained so that enough general features applicable to both seen
and unseen instances can be extracted; there may be undesirable effects due to over
training the network.

Advantages:
• Though genetic algorithms do not guarantee to find global optimum solution, they are good
in quickly finding good acceptable solutions.
• Hybridization of BPN with genetic algorithm is expected to provide many advantages
compared to what they alone can.
Genetic Algorithm and Neural Network
The figure illustrates a hybrid system combining a genetic algorithm (GA) with an
artificial neural network (ANN) to optimize the neural network's structure.

1. Initial Population: A set of neural network structures is generated, each representing

a potential solution.
2. Fitness Calculation: The ANN evaluates each structure's performance, likely using a
validation dataset.
3. Selection: The best-performing structures are selected based on a fitness criterion.
4. Reproduction, Crossover, Mutation: Genetic operators are applied to create new
neural network structures.
5. New Population after Generation: A new set of ANNs is formed from the previous
steps.
6. Stopping Condition: The process repeats until a stopping criterion is met (like a set
number of generations or a performance threshold).
7. Properties & Best individual ANN topology and ANN parameters: The best-
performing neural network structure and its parameters are identified.
8. ANN-GA Model (Prediction): The hybrid model is used for predictions or tasks.
9. Optimization (GA): The GA optimizes the parameters of the selected neural network
structures.
10. Selection of Clusters: This might refer to choosing groups of similar network
structures or parameters.

This system benefits from the GA's ability to search a large parameter space and the
ANN's capacity for modeling complex relationships.
 Genetic Algorithm Based Back-Propagation Network (BPN)

The process begins by encoding the neural network's weights as chromosomes within a GA. An initial
population of these chromosomes is randomly generated, representing various possible weight
configurations. The fitness of each chromosome is assessed based on the neural network's performance
with those weights. GA operators such as selection, crossover, and mutation are applied to evolve the
population towards better solutions. This cycle continues until the network's performance meets the
stopping criteria, at which point the optimal weights are determined. The approach leverages the GA's
ability to explore the solution space efficiently, improving the BPN's weight optimization process.

• Coding
• Weight Extraction
• Fitness Function
• Reproduction of offspring
• Convergence
 Genetic Algorithm Based Back-Propagation Network (BPN)
• Coding
 Genetic Algorithm Based Back-Propagation Network (BPN)

• Weight Extraction
 Genetic Algorithm Based Back-Propagation Network (BPN)

• Fitness Function
 Genetic Algorithm Based Back-Propagation Network (BPN)

• Fitness Function…
 Genetic Algorithm Based Back-Propagation Network (BPN)

• Reproduction of Offspring
 Genetic Algorithm Based Back-Propagation Network (BPN)

• Convergence
Advantages of Neuro-Genetic Hybrid Systems
Example of Neuro-Genetic Hybrid Systems
Example of Neuro-Genetic Hybrid Systems..
Example of Neuro-Genetic Hybrid Systems..
Example of Neuro-Genetic Hybrid Systems…
Example of Neuro-Genetic Hybrid Systems…
Example of Neuro-Genetic Hybrid Systems…
Example of Neuro-Genetic Hybrid Systems…
Example of Neuro-Genetic Hybrid Systems…
Example of Neuro-Genetic Hybrid Systems…
Example of Neuro-Genetic Hybrid Systems…
Example of Neuro-Genetic Hybrid Systems…
Example of Neuro-Genetic Hybrid Systems…
 Example of Neuro-Genetic Hybrid Systems…

Figure illustrates a sample selection of parents for the application of the two-point crossover
operator to produce offspring chromosomes. Here, the parents are selected in pairs at
random.
 Example of Neuro-Genetic Hybrid Systems…

The cross-sites of the chromosome parent pairs are randomly determined for each pair as
shown as Fig.
 Example of Neuro-Genetic Hybrid Systems…

The genes are exchanged as shown in Fig.

We call the new population P1. P1 comprises 40 chromosomes which are the offspring of the
earlier population generation P0.
 Example of Neuro-Genetic Hybrid Systems…

Convergence
• For any problem, if the GA is correctly implemented, the population evolves over successive
generations with the fitness value increasing towards the global optimum. Convergence is the
progression towards increasing uniformity. A population is said to have converged when 95% of
the individuals constituting the population share the same fitness value.

• The population P1 now undergoes the process of selection, reproduction, and crossover. The
fitness values for the chromosomes in P1 are computed, the best individuals replicated and
reproduction carried out using two-point crossover operator to form the next generation P2 of
chromosomes. The process of generation proceeds until at one stage 95% of the chromosomes
in the population Pi converge to the same fitness value. At that stage, the weights extracted
from the population Pi are the final weights to be used by the BPN.