VISVESVARAYA TECHNOLOGICAL UNIVERSITY

“Jnana Sangama”, Belagavi – 560 018.

TECHNICAL SEMINAR REPORT

ON

“GENETIC PROGRAMMING”
Submitted in partial fulfillment for the award of

Bachelor of Engineering in Computer Science and Engineering

Submitted By

THRIPURASRI S- 1CK20CS057
Under the guidance
Mr. RAJA A
Assoc Professor
Dept of CSE, CBIT-Kolar

C. BYREGOWDA INSTITUTE OF TECHNOLOGY

An ISO 9001:2015 Certified Institute
DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING
Kolar – Srinivaspur Road,
Kolar – 563101
 2023-2024

C BYREGOWDA INSTITUTE OF TECHNOLOGY

An ISO 9001:2015 Certified Institute

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Kolar-Srinivasapura Road,

Kolar-563101

CERTIFICATE

This is to certify that the technical seminar entitled “GENETIC PROGRAMMING” is a

bonafide work carried out by THRIPURASRI S bearing USN:1CK20CS057 in partial
fulfillment for the award of Bachelor of Engineering in Computer Science & Engineering of
the Visvesvaraya Technological University, Belagavi during the year 2023-24. It is certified
that all corrections/suggestions indicated for the internal assessment have been incorporated
in the report. The Technical Seminar report has been approved as it satisfies the academic
requirement in respect of technical seminar work prescribed for the VIII Semester Bachelor
of Engineering Degree.

_______________________ _______________________ _______________________

Signature of Guide Signature of Co-ordinator Signature of HOD
Mr. Raja A Mrs. Manjula S S Mr. VASUDEVA R
Assoc Professor, Asst Professor, Assoc Professor & HOD
Dept.of CSE, CBIT Dept.of CSE, CBIT Dept.of CSE, CBIT
 ABSTRACT
Genetic Programming (GP) is a computational technique inspired by the principles
of natural selection and genetics. It employs an evolutionary algorithm to automatically
generate computer programs to solve problems. In GP, candidate solutions are represented as
trees, with each node representing an operation or a value. These trees undergo a process of
evolution, where they are subjected to selection, crossover, mutation, and reproduction,
mimicking the process of natural selection.

The process begins with an initial population of randomly generated programs.

These programs are evaluated based on their fitness, which measures how well they solve the
given problem. The fittest programs are selected to produce offspring through genetic
operators such as crossover and mutation. Crossover involves exchanging subtrees between
parent programs, while mutation introduces random changes in the offspring's structure. The
new generation of programs undergoes evaluation, and the process iterates until a termination
condition is met, typically when a satisfactory solution is found or a maximum number of
generations is reached.

GP has been successfully applied in various domains, including symbolic regression,

classification, and optimization. Its ability to automatically evolve solutions makes it
particularly useful for problems where traditional algorithmic approaches are impractical or
inefficient.
 DECLARATION
I, THRIPURASRI S bearing USN 1CK20CS057, student of 8th semester B.E.,
Computer Science and Engineering of VTU, declare that this technical seminar report entitled
“GENETIC PROGRAMMING” embodies technical seminar work carried under the
guidance of Mr. RAJA A, Assoc Professor, Dept. Of CSE, CBIT as partial fulfilment of the
requirement of the award of degree in Bachelor of Engineering Computer Science and
Engineering, affiliated to Visvesvaraya Technological University, Belagavi during the
academic year 2023-24, Further the content embodies in the technical seminar report
has not been submitted previously by anybody for the award of any other degree.

PLACE: KOLAR SIGNATURE OF STUDENT

DATE: THRIPURASRI S

II
 ACKNOWLEDGEMENT
The completion of any work is a showcase of constant dedication and co-operation of
many people who lent their hands which went seen or unseen.

I am grateful to our Institution and Management, C. BYREGOWDA INSTITUTE OF

TECHNOLOGY with its ideals and inspirations for having provided us with the facilities,
which has made this project a success.

I would like to thank our beloved Assoc Professor and HOD Dr. VASUDEVA R,
Department of Computer Science & Engineering, for giving us guidance, valuable advice
and support.

I extend my gratitude to our guide Mr. RAJA A Assoc Professor, Department of

Computer Science & Engineering, for his valuable advice, support and constructive
suggestions.

I extend my gratitude to Technical Seminar Coordinator Prof. MANJULA S S ,

Assistant Professor, Department of Computer Science & Engineering, for her valuable
advice, support and constructive suggestions.

I also thank to all our professors and the entire department of Computer Science &
Engineering, for their co-operation and suggestions.

The report would be incomplete if we do not thank our parents and friends for their
continuous encouragement and moral support.

THRIPURASRI S - 1CK20CS057

III
 TABLE OF CONTENTS
ABSTRACT I
DECLARATION II
ACKNOWLEDGEMENT III
TABLE OF CONTENTS IV
LIST OF FIGURES VI

CH CHAPTER NAME PAGE NO

APTER NO
1 INTRODUCTION 1
1.1 Organization of report 2
2 LITERATURE SURVEY 3
3 REPRESENTATION, 6
INITIALIZATION OF PARENT
AND OPERATORS
3.1 Representation 6
3.2 Initialization of Population 7
3.3 Selection 8
3.4 Recombination and Mutation 8
4 WORKING AND ALGORITHMS 10
4.1 Working 10
4.2 Algorithms 10
4.2.1 Algorithm 1 11
4.2.2 Algorithm 2-Full and Grow 12
4.2.3 Algorithm 3-Interpreter of GP 12
CHAPTER 5 EXPERIMENTAL RESULTS AND 13
ANALYSIS
5.1 Mutation Method 13
5.2 Mutation Rate 14

III
 5.3 Crossover Method 15
5.4 Selection Method 16
CHAPTER 6 APPLICATIONS 17
6.1 Curve fitting, Data Modelling and 17
Symbolic Representation.
6.2 Financial Trading, Time Series 17
Prediction and Economic Modelling
6.3 Medicine, Biology and 17
Bioinformatics
6.4 Mixing GP with other Techniques 18
6.5 GP to Create Searchers and Solvers- 18
Hyper-Heuristics
6.6 Artistic 18
6.7 Entertainment and Computer Games 18
CHAPTER 7 CONCLUSION 19
REFERENCES 20

III
 LIST OF FIGURES
Figure no Figure name Page no
3.1 GP Syntax tree representation max(x*x,x+3y) 6
3.2 Full Initialization Method 7
3.3 Grow Initialization Method 7
3.4 Example for subtree crossover 8
3.5 Example for subtree mutation 9
4.1 General working of Genetic Programming 10
5.1 Average Fitness for different Mutation operators 13
5.2 Average Fitness overtime for different Mutation Rates 15
5.3 Average Fitness scores overtime for Crossover techniques 15
5.4 Selection methods: Comparison of Proportional solution 16
and elitism selection operators

VI
CHAPTER 1

INTRODUCTION
Genetic Programming (GP) is a powerful computational technique rooted in the
principles of natural evolution and genetics. It is a subfield of evolutionary computation that
aims to automatically evolve computer programs to solve complex problems. GP treats
computer programs as evolving entities represented in a tree-like structure, where each node
represents an operation or a value.

The process of GP begins with the initialization of a population of random programs.

These programs are typically generated using a predefined set of functions and terminals.
Functions represent operations like addition, subtraction, multiplication, etc., while terminals
represent constants or variables. The initial population represents potential solutions to the
problem at hand.

Once the initial population is created, the evolutionary process begins. This process
consists of several iterative steps, often referred to as generations. In each generation, the
fitness of each program in the population is evaluated. Fitness represents how well the
program solves the problem or achieves the desired objective. The evaluation function is
problem-specific and can vary depending on the application.

After evaluation, programs are selected to undergo genetic operations such as

crossover and mutation. Crossover involves swapping subtrees between selected programs,
creating offspring that inherit characteristics from both parents. Mutation introduces random
changes to the structure or parameters of selected programs, adding diversity to the
population.

The offspring generated through genetic operations form the next generation of
programs. This new generation undergoes evaluation, selection, crossover, and mutation,
iteratively evolving towards better solutions. The process continues until a termination
condition is met, such as finding a satisfactory solution or reaching a maximum number of
generations.

GP has been successfully applied in various domains, including symbolic regression,

classification, optimization, and automated problem-solving. Its ability to automatically
generate and evolve programs makes it particularly well-suited for problems with complex,
high-dimensional search spaces where traditional algorithmic approaches struggle.

1
GENETIC PROGRAMMING INTRODUCTION

In summary, Genetic Programming is a versatile and powerful technique for

automatic program generation and problem-solving, inspired by the principles of natural
evolution. By harnessing the evolutionary process, GP can produce solutions to a wide range
of challenging problems in a variety of fields.

1.1 Organization of the report

Chapter 1: This chapter provides the introduction to Genetic Programming.

Chapter 2: This chapter provides the information about the literature survey carried out for
this topic genetic programming.

Chapter 3: This chapter provides the detailed Representation, Initialization of parent and
operators used to construct the tree for genetic programming.

Chapter 4: This chapter provides the information about the working and algorithms used in
the genetic programming.

Chapter 5: This chapter provides the information about the experimental results and analysis
made on the genetic programming.

Chapter 6: This chapter describes the applications of genetic programming in various fields.

Chapter 7: Conclusion

REFERENCES

DEPT OF CSE, CBIT-KOLAR 2 2023-24

CHAPTER 2

LITERATURE SURVEY
Genetic Programming (GP) has been the subject of extensive research and
application across various domains, showcasing its versatility and effectiveness in solving
complex problems. Here's a literature survey highlighting key studies, advancements, and
resources related to GP:

[1] A Field Guide to Genetic Programming: By Poli, Langdon, and McPhee: This
comprehensive book provides an in-depth introduction to GP, covering fundamental
concepts, algorithmic details, and applications. It serves as a valuable resource for
both beginners and advanced practitioners. A Field Guide to Genetic Programming"
serves as an invaluable resource for researchers, practitioners, and students interested
in understanding and applying GP techniques to solve challenging problems in
various domains. It offers a comprehensive overview of the field, accompanied by
practical insights and guidance for implementing and deploying GP algorithms
effectively.
[2] Genetic Programming Theory and Practice: edited by Riolo, Soule, and Worzel:
This edited volume presents a collection of papers that delve into theoretical aspects,
practical applications, and recent developments in GP. It offers insights into the
theoretical underpinnings and practical considerations of GP. In classification tasks,
GP can be used to evolve programs that classify data into different categories. The
individuals in the population represent candidate solutions, and each individual is
evaluated based on how well it classifies the training data. The fitness function guides
the evolution process by rewarding individuals that produce more accurate
classifications.
[3] Advancements in Symbolic Regression with Genetic Programming: Symbolic
regression using GP has been a prominent area of research. Studies by Vladislav leva
et al. (2008) and Nguyen et al. (2015) have introduced innovative techniques and
strategies for improving the accuracy and efficiency of symbolic regression with GP.
[4] GP Applications in Classification and Prediction: GP has been widely applied in
classification tasks, ranging from pattern recognition to bioinformatics.

a. studies by Castelli and Manzoni (2002) and Orriols-Puig et al. (2009)

demonstrate the effectiveness of GP in generating accurate classifiers.

3
[5] GP for Optimization and Control: GP has also found applications in optimization
problems and control systems. Research by Koza (1992) and Bäck et al. (1997)
demonstrates the efficacy of GP in evolving optimal solutions for a wide range of
optimization tasks.
[6] GP in Real-World Applications: Numerous studies demonstrate the practical
applicability of GP in solving real-world problems across various domains. Notable
examples include applications in finance, engineering, and healthcare. For instance,
research by Cegielski et al. (2015) explores the use of GP for financial forecasting,
while studies by Kordon et al. (2011) and Koza et al. (2012) showcase GP's. Hybrid
and Ensemble Approaches: Researchers have explored hybridizing GP with other
machine learning techniques or forming ensembles to enhance performance and
robustness. Studies by Espejo et al. (2002) and Lopes et al. (2008) investigate the
effectiveness of hybrid and ensemble approaches in improving GP's performance.
[7] GP for Automated Machine Learning (AutoML): GP has emerged as a promising
technique for automating the machine learning pipeline, including feature
engineering, model selection, and hyperparameter tuning.
[8] Theoretical Advances and Extensions: Researchers continue to explore theoretical
aspects of GP and develop extensions to improve its capabilities. Studies by Fiduccia
and Mattheakis (2001) and Fenton and Nielsen (2016) introduce theoretical
frameworks and extensions to GP, addressing issues such as bloat control and multi-
objective optimization.
[9] Open-Source GP Frameworks and Libraries: Several open-source GP frameworks
and libraries are available, facilitating experimentation, research, and application
development. Examples include DEAP (Distributed Evolutionary Algorithms in
Python), ECJ (Evolutionary Computation in Java), and lil-gp (a small, portable
implementation of GP in C).
[10] GP for Evolving Neural Network: Genetic Programming has been extensively
utilized for evolving neural network architectures and weights. Research by Stanley
and Miikkulainen (2002) introduced NEAT (Neuroevolutionary of Augmenting
Topologies), a method that evolves both the structure and weights of neural networks
through GP, leading to the creation of more efficient and adaptive networks.

3
GENETIC PROGRAMMING Literature Survey

[11] GP for Automated Design and Invention: Genetic Programming has been applied
to automate the design and invention of novel solutions to engineering and design
problems. The work by Hornby et al. (2003) showcases the use of GP in evolving
complex designs, leading to innovative solutions in various domains, including
robotics and aerospace engineering.
[12] GP in Bioinformatics and Computational Biology: Genetic Programming has
shown promise in addressing challenges in bioinformatics and computational biology,
such as gene expression analysis and protein structure prediction. Research by Koza
et al. (2004) explores the application of GP in evolving programs for predicting gene
expression levels.

DEPT OF CSE, CBIT-KOLAR 5 2023-24

CHAPTER 3

Representation, Initialization and Operators in Tree

Based GP
In this Section we will introduce the basic tools and terms used in genetic
programming. In particular, we will look at how solutions are represented in most GP
systems (Sect.3.1), how one might construct the initial, random population (Sect.3.2), and
how selection (Sect.3.3) as well as recombination and mutation (Sect.3.4) are used to
construct new individuals.

3.1 REPRESENTATION
In GP programs are usually expressed as syntax trees rather than as lines of code. Figure 3.1
shows, for example, the tree representation of the program max(x*x,x+3*y). Note how the
variables and constants in the program x, y, and 3), called terminals in GP, are leaves of the
tree, while the arithmetic operations (+, *, and max) are internal nodes (typically called
functions in the GP literature). The sets of allowed functions and terminals together form the
primitive set of a GP system.

max

+ +

x x x ∗

3 y

Fig 3.1: GP syntax tree representing max(x*x,x+3y)

In more advanced forms of GP, programs can be composed of multiple components
(say, subroutines). In this case the representation used in GP is a set of trees (one for each
component) grouped together under a special root node that acts as glue, as illustrated in
Fig.3.1 We will call these (sub)trees branches. The number and type of the branches in a
program, together with certain other features of the structure of the branches, form the
architecture of the program.

6
3.2 Initialization of Population
Similar to other evolutionary algorithms, in GP the individuals in the initial
population are randomly generated. There are a number of different approaches to generating
this random initial population. Here we will describe two of the simplest (and earliest)
methods (the Full and Grow methods), and a widely used combination of the two known as
Ramped half-and-half. In both the Full and Grow methods, the initial individuals are
generated subject to a pre-established maximum depth. In the Full method (so named because
it generates full trees) nodes are taken at random from the function set until this maximum
tree depth is reached, and beyond that depth only terminals can be chosen. Figure 3.2 shows
snapshots of this process in the construction of a full tree of depth 2. The children of the *
node for example,

t=1 t=2 t=3 t=4

+ + + +

∗ ∗ ∗

x x y
t=5 t=6 t=7
+ + +

∗ / ∗ / ∗ /

x y x y 1 x y 1 0

Fig 3.2: Full Initialization method

+ + +

x x −

t=4 t=5
+ +

x − x −

Fig 3.3: Grow Initialization method

must be leaves, or the resulting tree would be too deep; thus at time t = 3 and time t = 4
terminals must be chosen (x and y in this case).

Where the Full method generates trees of a specific size and shape, the Grow
method allows for the creation of trees of varying size and shape. Here nodes are selected
from the whole primitive set (functions and terminals) until the depth limit is reached, below
which only terminals may be chosen (as is the case in the Full method). Figure 3.3 illustrates
this process for the construction of a tree with depth limit 2. Here the first child of the root +

6
node happens to be a terminal, thus closing off that branch before actually reaching the depth
limit. The other

6
GENETIC PROGRAMMING Representation ,initialization & operators

child, however, is a function (-), but its children are forced to be terminals to ensure that the
resulting tree does not exceed the depth limit.

3.3 Selection
Like in most other EAs, genetic operators in GP are applied to individuals that are
probabilistically selected based on fitness. That is, better individuals are more likely to have
more child programs than inferior individuals. The most commonly employed method for
selecting individuals in GP is tournament selection, followed by fitness-proportionate
selection, but any standard EA selection mechanism can be used. Since selection has been
described elsewhere in this book (in the Chapters on EAs), we will not provide any additional
details.

3.4 Recombination and Mutation

Where GP departs significantly from other EAs is in the implementation of the
operators of crossover and mutation. The most commonly used form of crossover is subtree
crossover. Given two parents, subtree crossover randomly selects a crossover point in each
parent tree. Then, it creates the offspring by replacing the sub-tree rooted at the crossover
point in a copy of the first parent with a copy of the sub-tree rooted at the crossover point in
the second parent, as illustrated in Fig.3.4.

Fig 3.4: Example of subtree crossover

Except in technical studies on the behaviour of GP, crossover points are usually not
selected with uniform probability. Typical GP primitive sets lead to trees with an average
branching factor of at least two, so the majority of the nodes will be leaves. Consequently,
the uniform selection of crossover points leads to crossover operations frequently exchanging

DEPT OF CSE, CBIT-KOLAR 8 2023-24

GENETIC PROGRAMMING Representation ,initialization & operators

only very small amounts of genetic material (that is, small subtrees); many crossovers may in
fact

DEPT OF CSE, CBIT-KOLAR 8 2023-24

GENETIC PROGRAMMING Representation ,initialization & operators

reduce to simply swapping two leaves. To counter this, Koza suggested the widely used
approach of choosing functions 90% of the time, while leaves are selected 10% of the time. .
Finally, it is worth mentioning that the notion of common region is related to the notion of
homology, in the sense that the common region represents the result of a matching process
between parent trees. It is then possible to imagine that within such a region transfer of
homologous primitives can happen in very much like the same way as it happens in GAs
operating on linear chromosomes. An example of recombination operator that implements
this idea is uniform crossover for GP

Mutation Mutation
+ Point + Point

+ 3 + ∗

x y x y y /
Randomly Generated
Sub-tree x 2
∗

y /

x 2

Fig 3.5: Example of subtree mutation.

The most commonly used form of mutation in GP (which we will call subtree
mutation) randomly selects a mutation point in a tree and substitutes the sub-tree rooted there
with a randomly generated sub-tree. This is illustrated in Fig.3.5. Subtree mutation is
sometimes implemented as crossover between a program and a newly generated random
program; this operation is also known as ‘headless chicken’ crossover [10]. Another common
form of mutation is point mutation, which is the rough equivalent for GP of the bit-flip
mutation used in GAs. In point mutation a random node is selected and the primitive stored
there is replaced with a different random primitive of the same arity taken from the primitive
set. If no other primitives with that arity exist, nothing happens to that node (but other nodes
may still be mutated). Note that, when subtree mutation is applied, this involves the
modification of exactly one subtree. Point mutation, on the other hand, is typically applied
with a given mutation rate on a per-node basis, allowing multiple nodes to be mutated
independently. While mutation is not necessary for GP to solve many problems, argues that,
in some cases, GP with mutation alone can perform as well as GP using crossover. While
mutation was often used sparsely in early GP work, it is more widely used in GP today,
especially in modelling applications.

DEPT OF CSE, CBIT-KOLAR 9 2023-24

CHAPTER 4

WORKING AND ALGORITHMS

4.1 WORKING
Fig 4.1 describes the General working of genetic programming.
 Crossover: the creation of one or two offspring programs by recombining randomly
chosen parts from two selected programs.
 Mutation: the creation of one new offspring program by randomly altering a
randomly chosen part of one selected program.
Solution
Generate Population Run Programs and
(* (SIN (- y x
))
of Random Programs Evaluate Their Quality (IF (> x 15.43)
(+ 2.3787 x)
(* (SQRT y)
(/ x 7.54))))

Breed Fitter Programs

Fig 4.1: General Working of Genetic Programming.

4.2 Algorithms
Genetic Algorithms (GAs) are a type of evolutionary algorithm inspired by the principles of
natural selection and genetics. They are used to solve optimization and search problems by
mimicking the process of evolution. GAs can be applied to a wide range of problems,
including optimization, combinatorial problems, and machine learning. Here's a detailed
definition and explanation of Genetic Algorithms:

 Basic Concept: Genetic Algorithms work by creating a population of candidate

solutions to a problem. Each candidate solution, often called a "chromosome,"
represents a potential answer to the problem. The algorithm evolves this population
through a series of iterations (or generations), gradually improving the quality of the
solutions.
 Chromosomes and Encoding: In GAs, chromosomes are typically represented as
strings of symbols, such as binary digits (bits), integers, or real numbers. The
encoding scheme defines how the problem variables are mapped to these strings.
 Fitness Function: A key component of GAs is the fitness function, which measures
the quality or performance of each chromosome. The fitness function determines how

10
GENETIC PROGRAMMING Working & Algorithms

well a candidate solution solves the problem, guiding the selection process during
evolution.
 Selection: Selection determines which chromosomes are chosen to create the next
generation. Common selection methods include roulette wheel selection, tournament
selection, and rank-based selection. The idea is to select chromosomes with higher
fitness, allowing them to pass on their genes to the next generation.
 Crossover: Crossover is a genetic operator that combines two parent chromosomes to
produce one or more offspring. It is designed to introduce new genetic material and
explore the solution space. Common crossover techniques include single-point, two-
point, and uniform crossover.
 Mutation: Mutation is another genetic operator that introduces random changes to
chromosomes, promoting diversity in the population. Mutation helps prevent
premature convergence to suboptimal solutions by exploring new areas of the solution
space.
 Generations and Termination: The GA process repeats over multiple generations.
Each generation involves selection, crossover, and mutation, leading to a new
population of candidate solutions. The algorithm terminates when a stopping criterion
is met, such as a maximum number of generations or an acceptable fitness level.

Overall, Genetic Algorithms offer a flexible and robust approach to solving

optimization and search problems, leveraging the principles of evolution to find high-quality
solutions.

4.2.1 Algorithm 1: Abstract GP algorithm

1. Randomly create an initial population of programs from the available primitives
2. repeat
3. Execute each program and ascertain its fitness.
4. Select one or two program from the population with a probability based on fitness to
participate in genetic operations .
5. Create new individual program by applying genetic operations with specified
probabilities.
6. until an acceptable solution is found or some other stopping condition is met (for
example, reaching a maximum number of generations).
7. return the best-so-far individual.

DEPT OF CSE, CBIT-KOLAR 11 2023-24

GENETIC PROGRAMMING Working & Algorithms

4.2.2 Algorithm 2: Pseudo code for recursive program generation with the Full and Grow
methods
Procedure:
gen rnd expr( func set, term set, max d, method )
1:method = grow and rand()then
1. set+func set
2. expr = choose random element( term set )
3. else
4. func = choose random element( func set )
5. for i = 1 to arity(func) do
6. arg i = gen rnd expr( func set, term set, max d - 1, method );
7. expr = (func, arg 1, arg 2, ...);
8. return expr
4.2.3 Algorithm 3: Typical interpreter for GP
1. if expr is a list then
2. proc = expr(1) {Non-terminal: extract root}
3. if proc is a function then
4. value = proc( eval(expr(2)), eval(expr(3)), ... ) {Function: evaluate arguments}
5.else
6. value = proc( expr(2), expr(3), ...) {Macro: don’t evaluate arguments}
7. else
8. if expr is a variable or expr is a constant then
9. value = expr {Terminal variable or constant: just read the value}
10.else
11. value = expr() {Terminal 0-arity function: execute}
12. return value

DEPT OF CSE, CBIT-KOLAR 12 2023-24

CHAPTER 5

EXPERIMENTAL RESULT AND ANALYSIS

In these experiments, the plug-in table given to the program is constructed in such a
way that it will be difficult to create an optimal solution. This was done by including a large
number4 of difficult plug-ins in the plug-in table. This prevents the fitness scores from
“clipping” at a maximum value of 30, which would make it difficult to compare results. For
more reasonable plug-in tables, the algorithm tends to arrive at solutions with fitness scores
of 30 for many different configurations of mutation method, mutation rate, and crossover
method.

5.1 Mutation Method

In the mutation method experiment, the two different mutation operators were
compared. The first mutation operator randomly increments and decrements a single gene,
while the second mutation operator swaps the values of two genes in the same chromosome.
This second operator was added because of the observation that switching the location of two
plug-ins can often improve network performance. Theprogramwastested under four
conditions. With the “classic” mutation operator only (pMutate = 0.1), with the swap
mutation operator only ( pMutateSwap = 0.1 ), with both mutation operators (pMutate =
pMutateSwap = 0.05) and with no mutation operators (pMutate = pMutateSwap = 0). For
each condition, the population size was set to 32 individuals, and was allowed to run for 100
generations. One-point crossover was used. Proportional selection was used. The average
fitness of all individuals in the population was used as a measure of success. For each of the
four configurations, four trials were conducted. The values from these trials were averaged to
give a single graph for each mutation operator, as shown in Fig 5.1.

13
Fig 5.1: Average Fitness for Different Mutation Operators

13
GENETIC PROGRAMMING Experimental Result & Analysis

Fig 5.1 has several interesting properties. First, it is apparent that the line
corresponding to no mutation eventually reaches a maximum fitness which it never exceeds.
This shows that not including mutation reduces the amount of exploration that is possible.
The genetic material present in the initial population will remain the same throughout
subsequent generations. It will only be combined in different ways through crossover
operations. Adding mutation operators allows exploration of new areas of the search space.

Second, the swap mutation operator did not perform better than the classical
mutation operator until very late in the simulation. This is probably because the swap
mutation operator does not explore as widely as the classical mutation operator, since it can
only change the location of genes which are already in the chromosome. The early wide
exploration provided by the classical mutation operator allowed it to quickly find better
solutions than the other three techniques.

5.2 Mutation Rate

Having determined that the combination of normal mutation and swap mutation
gives better performance in this problem domain, the effect of changing the probability of
mutation will be considered. The two probabilities pMutate and pMutateSwap will be set to
the same value. This value, p, will be set to 0, 0.05, 0.10, 0.15, and 0.50. One-point crossover
will be used. Four trials for each mutation rate will be performed, and the average fitness
from each generation in each of the four trials will be recorded. The experiments are run with
the same plug-in configuration used before, so it is difficult for the algorithm to produce a
chromosome with perfect fitness. The results of these experiments are shown in Figure 5.2.
Clearly, the selection of the mutation rate affects the quality of the chromosomes produced by
the genetic algorithm. The worst performance was achieved with p = .5, while the best
performance was achieved with p = .1. Since the best performance was achieved by a
mutation rate in the middle of the test values, values above and below this value should be
considered separately. Those mutation values below p = .1 (p = 0,p = 0.05) produced worse
performance. This can be attributed to reduced exploration of the search space because of less
frequent mutation. In the case of p =0, the algorithm eventually converged on a single
solution with a relatively low fitness. Without mutation, it was not possible to explore further
solutions through crossover operations only. One way to address this problem would be to
make the probability of mutation depend on the fitness of the individual, to reduce the chance
that good solutions will be destroyed by excessive mutation. Increasing the population size

DEPT OF CSE, CBIT-KOLAR 14 2023-24

GENETIC PROGRAMMING Experimental Result & Analysis

might also help in this situation, since it will increase the probability that more good solutions
survive.

Fig 5.2 : Average Fitness Over Time For Different Mutation Rates

5.3 Crossover Method

The genetic algorithm can produce two offspring from two parents using one of
three crossover techniques. The algorithm will be run using pMutate = pMutateSwap = 0.1, a
population of 32 individuals, and 100 generations. For each of the crossover operations, four
trials will be conducted. The average fitness score of the population over time will be
averaged for each of these four trials, producing three graphs. These graphs will be compared
to determine the best crossover strategy for this problem domain. Figure 5.3 shows the
average fitness score of a populations over time given different crossover techniques.
Uniform crossover produced the best result in the end, and showed a slow, steady
development of the population from lower fitness to higher fitness over time.

DEPT OF CSE, CBIT-KOLAR 14 2023-24

GENETIC PROGRAMMING Experimental Result & Analysis

Fig 5.3: Average fitness scores over time for different crossover techniques

DEPT OF CSE, CBIT-KOLAR 14 2023-24

GENETIC PROGRAMMING Experimental Result & Analysis

5.4 Selection Method

Two methods of selecting which chromosomes to use for reproduction are
compared. Proportional selection bases the probability of selection on the fitness score, with
fitter individuals being more likely to be selected. Elitism selection selects the fittest half of
the population, completely excluding the least fit individuals. In general, elitism can often
restrict the search space because it prevents genetic information from less fit individuals from
being combined with other genetic information, potentially giving more fit individuals.
However, in this case, elitism turned out to be the best selection method. In this case, elitism
clearly out-performs proportional selection. Elitism exerts a much stronger selective pressure
on the population, giving a much steeper rise in average fitness over time. Eventually, the
population reached a maximum average fitness of slightly under 30. This below Fig 5.4
shows the greater selection pressure imposed by elitism. However, this selection pressure
comes at the price of decreased genetic diversity. In this problem domain, there are a large
number of valid solutions, so the increased search-space exploration provided by this
diversity might not be necessary to find a good solution.

Fig 5.4 : Comparison of proportional selection and elitism selection operators

DEPT OF CSE, CBIT-KOLAR 16 2023-24

CHAPTER 6

APPLICATIONS
Since its early beginnings, GP has produced a cornucopia of results. The literature,
which covers more than 5000 recorded uses of GP, reports an enormous number of
applications where GP has been successfully used as an automatic programming tool, a
machine learner or an automatic problem-solving machine.

6.1 Curve Fitting, Data Modelling, and Symbolic Regression

In principle, the possible applications of GP are as many as the applications for
programs (virtually infinite). However, before one can try to solve a new problem with GP,
one needs to define an appropriate fitness function. In problems where only the side effects of
the program are of interest, the fitness function usually compares the effects of the execution
of a program in some suitable environments with a desired behaviour, often in a very
application dependent manner. In many problems, however, the goal is finding a function
whose output has some desired property – for example, it matches some target values (as in
the example given in Sect.4), or it is optimum against some other criteria. This type of
problem is generally known as a symbolic regression problem.

6.2 Financial Trading, Time Series Prediction and Economic Modelling

The efficient markets hypothesis is a tenet of economics. It is founded on the idea
that everyone in a market has ‘perfect information’ and acts ‘rationally’. If the efficient
markets hypothesis held, then everyone would see the same value for items in the market and
so agree the same price. Without price differentials, there would be no money to be made
from the market itself. Whether it is trading potatoes in northern France or dollars for yen it is
clear that traders are not all equal and considerable doubt has been cast on the efficient
markets hypothesis. So, people continue to play the stock market. Game theory has been a
standard tool used by economists to try to understand markets but is increasingly
supplemented by simulations with both human and computerized agents. GP in increasingly
being used as part of these simulations of social systems.

6.3 Medicine, Biology and Bioinformatics

GP has long been applied to medicine, biology and bioinformatics. Early work by
Handley and Koza used GP to make predictions about the behavior and properties of

17
biological systems, principally proteins. Oakley, a practising medical doctor, used GP to
model blood flow in toes as part of his long term interests in frostbite.

17
6.4 Mixing GP with Other Techniques
GP can be hybridised with other techniques. Iba, Nikolaev, and Zhang have
incorporated information theoretic and minimum description length ideas into GP fitness
functions to provide a degree of regularization and so avoid over-fitting . As mentioned
computer language grammars can be incorporated into GP. Indeed Wong [has had success
integrating these with GP. The use of simulated annealing and hill climbing to locally fine
tune parts of solutions found by GP was described.

6.5 GP to Create Searchers and Solvers – Hyper-Heuristics

Hyper-heuristics could simply be defined as ‘heuristics to choose other heuristics’.
A heuristic is considered as a rule-of-thumb or ‘educated guess’ that reduces the search
required to find a solution. The difference between meta-heuristics and hyper-heuristics is
that the former operate directly on the problem search space with the goal of finding optimal
or near-optimal solutions. The latter, instead, operate on the heuristics search space (which
consists of the heuristics used to solve the target problem). The goal then is finding or
generating high-quality heuristics for a problem, for a certain class of instances of a problem,
or even for a particular instance.

6.6 Artistic
Computers have long been used to create purely aesthetic artifacts. Much of today’s
computer art tends to ape traditional drawing and painting, producing static pictures on a
computer monitor. However, the immediate advantage of the computer screen – movement –
can also exploited. In both cases EC can and has been exploited. Indeed with evolution’s
capacity for unlimited variation, EC offers the artist the scope to produce ever changing
works. Some artists have also worked with sound.

6.7 Entertainment and Computer Games

Today the major usage of computers is interactive games. There has been a little
work on incorporating artificial intelligence into mainstream commercial games information
on how they use AI. Work on GP and games includes [16,389]. After chairing the IEEE
Symposium on Computational Intelligence and Games 2005 at Essex University, Lucas
founded the IEEE Computational Intelligence Society’s Technical Committee on Games. GP
features heavily in the Games TC’s activities, for example Othello, Poker, Backgammon,
Draughts, Chess, Ms Pac-Man, robotic football and radio controlled model car racing.

DEPT OF CSE, CBIT-KOLAR 18 2023-24

CHAPTER 7

CONCLUSION
The most difficult part of this project was the development of the fitness function. It
required modelling a complex system, and is based on a number of simplifying assumptions,
particularly with respect to the network. The utility of the solutions generated by this
algorithm could be improved by refining the fitness function so that it more closely reflects
the actual performance possible in the video processing network. However, based on the data
I was able to collect, the fitness function models the network reasonably well. For the
purpose of exploring genetic algorithms, the fitness function provides a complex feature
space with conflicting goals. I suspect that this made it easy to see the difference between
some of the configurations that I tried in my experiments. For example, the difference in
behaviour of elitism selection and proportional selection was very apparent. The comparison
of various mutation methods, including one which was created because I suspected it would
be useful for this problem, was interesting. It showed that not using mutation limited the
maximum fitness that could be achieved, while using both mutation operators in concert gave
better results than either one in isolation. In this case, increased searching gave better results.
When different mutation rates were compared, however, it became apparent that too much
searching could also be a problem.

A mutation rate which is too high has a tendency to destroy information contained
in successful individuals. The examination of one-point, two-point and uniform crossover
also showed that this can have an effect on the performance of the genetic algorithm.
However, it was not clear from the data I was able to collect which of the crossover
techniques is the best to use in th is case. This could be an area for further exploration.
Finally, the most surprising result was that elitist selection gave better results than
proportional selection in less time. The conventional wisdom seems to be that elitism limits
the ultimate success of the algorithm by excluding useful genetic material, but in this case the
increased selection pressure seems to have been more important.

19
 REFERENCES
[1] Poli, Langdon, and McPhee, “A Field Guide to Genetic Programming”., published
year-2008[Link to the book](https://www.cs.miami.edu/home/poli/GECCO2008/).
[2] Riolo,Soule and Worzel edited “Genetic Programming Theory and Practice”.
[3] Vladislav leva and Nguyen published “Advancements in Symbolic Regression with
Genetic Programming” in the year 2015.
[4] Castelli, M., & Manzoni, L. (2002). “Genetic programming and feature construction
for classification tasks”. Proceedings of the 2002 Congress on Evolutionary
Computation (CEC).
[5] Research by Koza (1992) and Bäck et al. (1997)., “GP for Optimization and
Control”, IEEE Transactions on Evolutionary Computation, 1997
[6] Cegielski, M., Kowalski, P. A., & Winkler, K. J. (2015). “Financial forecasting using
genetic programming: A survey of the literature”. Expert Systems with Applications in
2015.
[7] Research by Kotthoff et al. (2017) and Orzechowski et al. (2017) investigates “GP for
Automated Machine Learning (AutoML)” in year 2017.
[8] Fenton, M., & Nielsen, T. D. (2016). Grammatical evolution for automated software
engineering: A critical review. “Genetic Programming and Evolvable Machines” in
2016 .
[9] “Open-Source GP Frameworks and Libraries”: Several open-source GP frameworks
and libraries are available, facilitating experimentation, research, and application
development. Examples include DEAP (Distributed Evolutionary Algorithms in
Python).
[10] Stanley, K. O., & Miikkulainen, R. (2002). Evolving neural networks through
augmenting topologies. “Evolutionary Computation”, 10(2), 99-127.[Link to the
paper] (http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf)
[11] Hornby, G. S., Lohn, J. D., & Linden, D. S. (2003). Computer-automated evolution of
an X-band antenna for NASA's space technology 5 mission. “Evolutionary
Computation.(https://www.mitpressjournals.org/doi/abs/10.1162/1063656033225181
71)
[12] Koza, J. R., Mydlowec, W., & Lanza, G. (2004). An empirical investigation of the
role of gene expression programming in the prediction of gene expression. “Genetic
Programming Theory and Practice II”,
(http://www.genetic-programming.org/hc2003/Koza2003GPTP.pdf).dss