Kerbala University Machin Learning

College of Computer Science and Information Technology

Department of Computer Science Dr. Elaf Adil – 8 Lecture

Soft Computing
2h Lecture Dr. Elaf Adil
Learning Objectives
The general objective of the lecture:
he primary objective of this lecture is to provide an in-depth understanding of Soft
Computing and Genetic Algorithms (GA), focusing on their definitions,
components, working principles, and real-world applications. The lecture aims to
highlight how soft computing techniques, particularly Genetic Algorithms, solve
complex optimization problems in various domains.
Detailed objectives:

At the end of the lecture, the students are expected to be able:

Understand Soft Computing – Learn what soft computing is and how it differs
from traditional computing.

Explore Soft Computing Components.

Learn Genetic Algorithms (GA) – Understand the working principles, including

selection, crossover, and mutation.

Apply GA in Problem-Solving – Demonstrate GA with a simple example and

solve the Knapsack problem.

The target audience of the lecture:

Undergraduate-stage (3rd stage) students in computer science.

• Lecture duration: 1 lecture.
• Lecture duration: 2 hours.
 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

1. Components of Soft Computing

Soft Computing is a collection of computational techniques that model complex
systems and human-like problem-solving abilities. Unlike traditional hard
computing, which relies on precise models and deterministic solutions, soft
computing deals with approximate reasoning, uncertainty, and learning from data.
Biological processes inspire it and provide flexible solutions to real-world problems
where exact mathematical formulations are difficult.

Key Features of Soft Computing:

• Deals with imprecision and uncertainty

• Tolerant to approximations

• Mimics human reasoning and adaptability

• Provides robust solutions for complex problems

2. Components of Soft Computing

Soft computing consists of several methodologies that work together to solve real-
world problems. The main components include:

a. Fuzzy Logic (FL)

b. Artificial Neural Networks (ANN)

c. Evolutionary Computation (EC)

d. Sport Vector Machine

 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

3. The Concept of Genetic Algorithms (GA) from Darwin's

Evolutionary Theory
The foundation of Genetic Algorithms (GA) is deeply rooted in Charles Darwin's
theory of natural selection and evolution, introduced in his 1859 book, On the Origin
of Species. Genetic Algorithms simulate the process of natural evolution to solve
optimization and search problems in computing.
Darwin's Theory of Evolution

Darwin’s theory of natural selection states that:

• Organisms compete for survival.

• Individuals with advantageous traits (higher fitness) are more likely to

survive and reproduce.

• These traits are passed to offspring, gradually improving the species over
generations.

• Over time, natural selection leads to the evolution of the fittest organisms.

This idea of selection, inheritance, and gradual improvement forms the basis of
Genetic Algorithms in artificial intelligence and optimization.

4. What is GA
➢ A genetic algorithm (or GA) is a search technique used in computing to find
true or approximate solutions to optimization and search problems.
➢ (GA)s are categorized as global search heuristics.
➢ (GA)s are a particular class of evolutionary algorithms that use techniques
inspired by evolutionary biology such as inheritance, mutation, selection, and
crossover (also called recombination).Hierarchical clustering is a method of
 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

cluster analysis that seeks to build a hierarchy of clusters. It can be classified

into two types: agglomerative and divisive.
➢ The evolution usually starts from a population of randomly generated
individuals and happens in generations.
➢ In each generation, the fitness of every individual in the population is
evaluated, multiple individuals are selected from the current population
(based on their fitness), and modified to form a new population.
➢ The new population is used in the next iteration of the algorithm.
➢ The algorithm terminates when either a maximum number of generations has
been produced, or a satisfactory fitness level has been reached for the
population.

Vocabulary

1. Individual - Any possible solution

2. Population - Group of all individuals

3. Fitness – Target function that we are optimizing (each individual has a fitness)

4. Trait - Possible aspect (features) of an individual

5. Genome - Collection of all chromosomes (traits) for an individual

Key Steps:

• Start with a large “population” of randomly generated “attempted

solutions” to a problem

• Repeatedly do the following:

❖ Evaluate each of the attempted solutions

❖ (probabilistically) keep a subset of the best solutions
 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

❖ Use these solutions to generate a new population

• Quit when you have a satisfactory solution (or you run out of time) Compute.

The conceptual description of the algorithm is stated in the following block:

5. What is a Fitness Function?

A fitness function is a crucial part of a genetic algorithm. It assesses the quality, or

"fitness," of candidate solutions (also known as individuals or chromosomes) to a
problem. The fitness function is used to rank individuals and helps the algorithm
decide which individuals to keep and which to discard during the selection phase.
Designing a Fitness Function
 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

Designing an effective fitness function is crucial for the success of your genetic
algorithm. Here are a few guidelines to follow when creating one:

Be Problem-Specific: Your fitness function should be tailored to the specific

problem you're trying to solve. It should capture the essence of the problem and
reflect the objectives you want to achieve.

Reward Good Performance: The fitness function should differentiate between

good and bad solutions, assigning higher fitness scores to better solutions, and lower
scores to worse solutions.

Avoid Local Optima: A good fitness function should help your genetic algorithm
escape local optima – a suboptimal solution that appears optimal within a limited
search space. Design your fitness function to encourage exploration of the search
space, avoiding premature convergence.

Be Efficient: The fitness function may be called thousands or millions of times

during the algorithm's execution. Ensure that it's computationally efficient to avoid
slowing down the algorithm.

6. Genetic Algorithm Operators

6.1.Rrepresentation of initial population

Each individual (solution) in the population is represented as a chromosome.

Common representations include:

• Binary Representation: A chromosome is a string of 0s and 1s.

 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

• Real-valued Representation: A chromosome consists of real numbers.

• Permutation Representation: Used for ordering problems (e.g., the

Traveling Salesman Problem).

• Tree Representation: Used in Genetic Programming.

 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

6.2 Population Size

• The number of individuals (N) in the population affects performance.

• A small population may converge quickly but risks premature convergence.

• A large population ensures diversity but increases computational cost.

• A typical range is 50 to 1000 individuals, depending on the problem.

6.3 Methods for Initial Population Generation

(A) Random Initialization

• The most common method.

• Individuals are generated randomly within the search space.

• Ensures diversity but may lead to poor initial solutions.

(B) Heuristic-Based Initialization

• Uses domain knowledge to generate individuals close to optimal solutions.

• Helps in faster convergence but may limit exploration.

 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

(C) Opposition-Based Initialization

• Improves diversity by considering both randomly generated individuals and

their opposite values in the search space.

(D) Latin Hypercube Sampling (LHS)

• Ensures better coverage of the search space compared to pure random

sampling.

• Especially useful for real-valued problems.

7. Methods of Selection

Selection methods are crucial for guiding the evolution of solutions toward
optimality. They determine which individuals from the current population are
chosen to reproduce and pass their genetic material to the next generation. Effective
selection increases the likelihood of propagating high-quality solutions, thereby
enhancing the algorithm's performance.

Here are some commonly used selection methods:

Roulette Wheel Selection

In a roulette wheel selection, the circular wheel is divided as described before. A

fixed point is chosen on the wheel circumference as shown and the wheel is rotated.
The region of the wheel which comes in front of the fixed point is chosen as the
parent. For the second parent, the same process is repeated.
 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

It is clear that a fitter individual has a greater pie on the wheel and therefore a greater
chance of landing in front of the fixed point when the wheel is rotated. Therefore,
the probability of choosing an individual depends directly on its fitness.

Stochastic Universal Sampling (SUS)

Stochastic Universal Sampling is quite similar to Roulette wheel selection, however

instead of having just one fixed point, we have multiple fixed points as shown in the
following image. Therefore, all the parents are chosen in just one spin of the wheel.
Also, such a setup encourages highly fit individuals to be chosen at least once.
 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

It is to be noted that fitness proportionate selection methods don’t work for cases
where the fitness can take a negative value

Tournament Selection

In K-Way tournament selection, we select K individuals from the population at

random and select the best out of these to become a parent. The same process is
repeated for selecting the next parent. Tournament Selection is also extremely
popular in literature as it can even work with negative fitness values.

Rank Selection

Rank Selection also works with negative fitness values and is mostly used when the
individuals in the population have very close fitness values (this happens usually at
the end of the run). This leads to each individual having an almost equal share of the
pie (like in case of fitness proportionate selection) as shown in the following image
and hence each individual no matter how fit relative to each other has an
approximately same probability of getting selected as a parent. This in turn leads to
 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

a loss in the selection pressure towards fitter individuals, making the GA to make
poor parent selections in such situations.

In this, we remove the concept of a fitness value while selecting a parent. However,
every individual in the population is ranked according to their fitness. The selection
of the parents depends on the rank of each individual and not the fitness. The higher
ranked individuals are preferred more than the lower ranked ones.

Chromosome Fitness Value Rank

A 8.1 1

B 8.0 4

C 8.05 2

D 7.95 6

E 8.02 3

F 7.99 5

Random Selection
 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

In this strategy we randomly select parents from the existing population. There is no
selection pressure towards fitter individuals and therefore this strategy is usually
avoided.
8. Methods of Reproduction

reproduction methods are essential for generating new candidate solutions

(offspring) from the current population. These methods aim to combine and modify
existing solutions to explore the search space effectively and converge toward
optimal or near-optimal solutions. The primary reproduction methods in GAs
include:

1. Crossover (Recombination):

o Purpose: Combines genetic information from two parent solutions to produce

offspring, promoting the inheritance of favorable traits.

o Common Techniques:

One-Point Crossover: A single crossover point is selected on the parent

chromosomes, and the segments beyond that point are swapped to create
offspring.

Two-Point Crossover: Two crossover points are selected, and the

chromosome segments between them are exchanged between parents.
 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

Uniform Crossover: Each gene is chosen randomly from one of the parents,
allowing for more diverse offspring.

2. Mutation:

Introduces random alterations to individual genes in a chromosome,

maintaining genetic diversity and enabling the algorithm to explore new areas
of the search space.

Example:

the MAXONE problem Suppose we want to maximize the number of

ones in a string of l binary digits Is it a trivial problem?

 Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

It may seem so because we know the answer in advance. However, we can think of
it as maximizing the number of correct answers, each encoded by 1, to l yes/no
difficult questions.

o An individual is encoded (naturally) as a string of l binary digits

o The fitness f of a candidate solution to the MAXONE problem is the number
of ones in its genetic code
o We start with a population of n random strings.
o Suppose that L= 10 and n = 6
We toss a fair coin 60 times and get the following initial population:
s1 = 1111010101 f (s1) = 7
s2 = 0111000101 f (s2) = 5
s3 = 1110110101 f (s3) = 7
s4 = 0100010011 f (s4) = 4
s5 = 1110111101 f (s5) = 8
s6 = 0100110000 f (s6) = 3
Step 1: Selection
We randomly (using a biased coin) select a subset of the individuals based
on their fitness:
Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

Step 2: crossover
➢ Next we mate strings for crossover. For each couple we first decide
(using some pre-defined probability, for instance 0.6) whether to
actually perform the crossover or not.
➢ If we decide to actually perform crossover, we randomly extract the
crossover points, for instance 2 and 5.

Crossover result

Before crossover:

s1 = 1111010101 s2 = 1110110101

After crossover:

s1= 1110110101 s2 = 1111010101

Step 3: mutations

The final step is to apply random mutations: for each bit that we are to copy
to the new population we allow a small probability of error (for instance 0.1)

Initial strings

Initial strings After mutating

s1= 1110110101 s1 = 1110100101
s2 = 1111010101 s2 = 1111110100
s3 = 1110111101 s3 = 1110101111
s4 = 0111000101 s4 = 0111000101
s5= 0100011101 s5 = 0100011101
s6= 1110110011 s6 = 1110110001
And now, iterate …
Kerbala University Machin Learning
College of Computer Science and Information Technology
Department of Computer Science Dr. Elaf Adil – 8 Lecture

In one generation, the total population's fitness changed from 34 to 37, thus
improving by ~9%.

We go through the same process all over again, until a stopping criterion is
met.

<Best Regards>

Dr. Elaf Adil