PID tuning using Machine Learning

medium.com/@maohar502/pid-tuning-using-machine-learning-6cf6f7fe5690

November 15, 2020

PID controller is one of the most popular closed-loop controllers which is used in the
automation industry. By fine-tuning 3 constants, you are able to achieve a system which is
almost free from any errors.

On the other hand, Genetic Algorithms are being heavily used by various machine
learning enthusiasts around the world for building an efficient and robust deep learning
network.

Through this article, I’ll try to explain as to how the fusion of these two can help us
achieve a nearly perfect PID controller. We’ll use a genetic algorithm to find the 3
constant values(Kp, Ki and Kd).

The structure of this article will be :

1. What are control systems?

2. What is a PID controller?
3. What are genetic algorithms?
4. Using GA for tuning a PID controller.

So without any further due, let’s get started!!

In simple terms, control system takes some sets of inputs, regulates them to derive the
desired output and then directs them. Control system is usually of two types: 1) Open-
loop and 2) Closed-loop. The only difference being that, in closed-loop, the error is sent
into the controller as a feedback signal. In this article, we’ll be focusing on closed-loop
control systems.

Skeleton of a closed-loop control system

1/7
2. What is a PID controller?

In a PID controller, we calculate an error e(t) as the difference between the desired set-
point and the current value(process variable) and pass it as a feedback signal. The error
e(t) is then corrected based on the proportional(p),integral(i) and derivative(d) terms.

A PID controller

Still not clear? Let’s take an example:

Let’s say you want a drone to fly at an altitude of 3.0m(desired setpoint) and you are
currently at a height of 1.0m from the ground(process variable i.e the current value). To
reach the desired set-point, you’ll have to change the speed of the propellers. This speed
will be calculated based upon the output that we get from the PID controller.

Here the error term e(t) = (3.0–1.0) = 2.0 .

Now, the proportional error = e(t),

the integral error = (e(t) + sum of all previous errors)*dt ,

and the derivative error = (e(t)-e(t-1))/dt

and now the propeller speed(prop_speed) will be modified as :

prop_speed=Kp*proportional_error+Ki*integral_error+Kd*derivative_error

This closed control system loop will help us to stabilise the drone at the desired set-point
of 3.0m . The error is continuously feed back into the system and prop_speed is set
accordingly.

(Note: For simple explanation, I’ve considered only one propeller. It can be replicated for
rest 3. Alternately, you can modify the value of the throttle as well.)

But how do these terms help us to reach the desired point? Let’s see.

1. The proportional term (Kp*proportional_error): helps us to reduce the rise time. In

simple terms, helps us to reach the desired set point of 3.0m quickly.

2/7
 2. The integral term(Ki*integral_error): helps us to reduce any steady-state error. In
practical life, we can never achieve an error-free system, and this term helps to
reduce it as much as possible. For e.g, if our drone has reached 3.3m, then the
derivative term will help to reduce the error and bring the drone to an altitude of 3.0
(+/- 0.0002)m.
3. The derivative term(Kd*derivative_error): helps us to prevents any overshoot i.e
prevents the drone to fly off to an altitude much away from the desired set point e.g
>4.0m

But how do we find the correct constant values(Kp, Ki and Kd)?

There are many ways to find these constants. In this article, we’ll see how to find the
optimal values using GA.

3. What are genetic algorithms?

The concept of Genetic algorithms is heavily inspired by Charles Darwin’s theory of

natural selection. It states that off-springs will be naturally selected in such a way that only
best set off-springs will survive in the consecutive generations. In other words, only the
best characteristics of the current generation will be passed onto the next generation.
GA’s are used in optimisation problems.

They comprise of the following steps:

1. Create an Initial population

2. Defining a fitness function
3. Selection
4. Crossover
5. Mutation

Creating an Initial Population:

We start by generating a set of random values. Each value in this is known as a gene. A
set of genes is referred to as a chromosome, and a bunch of chromosomes create a
population.

3/7
 Credits : Vijini Mallawaarachchi

Fitness Function:

A fitness function is defined to measure the fitness of each chromosome. Speaking in

terms of deep learning, the cost function J is the fitness function. Based on the fitness,
the probability of the chromosome being selected in the next generation is calculated.
The higher the fitness, the higher the probability to be selected in the next generation.
Fitness can be defined according to your need. For e.g: In an optimisation problem if you
want to minimise the error, you’ll define a function which will be equal to 0.

Selection:

The main idea of selection is to select/generate individuals for the next generation. There
a lot of methods for selection, few of them are:

i. Elitism: Selecting the fittest individual directly to the next generation

ii. Replication: Creating a copy of a chromosome and using it in the next generation

iii. Crossover: In crossover, we select a bunch of individuals and select a few genes from
each of the chromosomes and exchange them

4/7
 Credits: Machine Learning Control by Steve Brunton

Mutation:
To ensure diversity and to prevent an early repetition of the chromosomes(pre-mature
convergence) we perform mutation for a few genes. In mutation, the bits are flipped.

Credits : Vijini Mallawaarachchi

After performing mutation, we end up generating a new set of population.

All of these 5 steps are continuously looped until we reach our stopping condition. The
stopping condition can either be when you reach your desired fitness or the set of
chromosomes keep on repeating in successive generations. You can define your own
stopping condition.

4. Using GA for tuning a PID controller.

Alright now that we have laid our foundation let’s see how to use GA to find the values
Kp, Ki and Kd.

Let’s consider the previous example of the drone. Our goal is to find the values of Kp, Ki
and Kd in such a way that the drone reaches the desired set-point of 3.0m in altitude as
fast as possible without any overshoot.

Our fitness function will be e(t) = 0 where the function e at any point t is the difference
between the present altitude and 3.0m i.e (3-current_altitude).

5/7
We’ll develop 3 generations and have a population of 5 in each generation. Our stopping
condition will be when we reach the last generation.

1st Generation:

Generating the 2nd generation from 1st

As you can see from the image, we start with a set of random values for our three
constants. We set these values in our fitness function and calculate the fitness(error).
After calculating the fitness we start our process of selection.

Since the 1st row(c1,c2,c3) had the lowest error, we directly select it to our 2nd
generation(elitism). After that, we applied a crossover between the 2nd row(d1,d2,d3) and
3rd row(a1,a2,a3) with a cross-over point of 1 . We applied the similar crossover between
the 2nd row(d1,d2,d3) and 4th row(b1,b2,b3). Both of these values gave us 4 more
values. With the help of these algorithms, we generated our 2nd generation.

2nd Generation:

Generating the 3rd generation.

We apply similar operations to generate the 3rd generation from the 2nd generation.

Note: You can develop your own algorithm to generate new generations. You can set
your own threshold value through which you want to apply elitism, replication, crossover
etc. A GA usually has a lot of generations with a greater population.

6/7
Now that we’ve reached our 3rd generation, our final generation, we stop developing new
generations. We’ll calculate the fitness of each chromosome and select the chromosome
having the best fitness. In our case, the PID values having the least error.

To summarise, by iterating through a bunch of generations containing a different set of

values and by propagating only the best values to the next generation we end up having
a near to perfect PID controller.

Now that you’ve seen how machine learning algorithms can be used to optimise PID
controller, do you wonder if the reverse is possible??
Well yes. There are a lot of researches that have explored this aspect as well. Here is one
such study: A PID Controller Approach for Stochastic Optimisation of Deep Networks-
https://openaccess.thecvf.com/content_cvpr_2018/papers/An_A_PID_Controller_CVPR_
2018_paper.pdf

References:

1. Introduction to Genetic Algorithm - Including example code: . In this article Vijini

Mallawaarachchi explains the concept of GA in simple and clear terms.
2. Machine Learning Control Overview by Steve Brunton: . This amazing series of
videos on machine learning control and PID optimisation will help you understand
the concept deeply (Bonus: There’s a Matlab code as well :P)

Congratulations if you’ve made this far. I hope this blog helped you to gain an insight on
how to use the machine learning technique of genetic algorithms to tune your PID
controller and you’ll use it.

7/7