International Journal of

Intelligent Systems and

Applications in Engineering Advanced Technology and Science

ISSN:2147-67992147-6799 www.atscience.org/IJISAE Original Research Paper

Artificial Bee Colony Algorithm Based Linear Quadratic Optimal

Controller Design for a Nonlinear Inverted Pendulum
Baris Ata1*, Ramazan Coban2
Received 13th September 2014, Accepted 08th December 2014
DOI: 10.1039/b000000x

Abstract: This paper presents a linear quadratic optimal controller design for a nonlinear inverted pendulum. Linear Quadratic Regulator
(LQR), an optimal control method, is usually used for control of the dynamical systems. Main design parameters in LQR are the weighting
matrices; however there is no relevant systematic techniques presented to choose these matrices. Generally, selecting weighting matrices
is performed by trial and error method since there is no direct relation between weighting matrices and time domain specifications like
overshoot percentage, settling time, and steady state error. Also it is time consuming and highly depends on designer’s experience. In this
paper LQR is used to control an inverted pendulum as a nonlinear dynamical system and the Artificial Bee Colony (ABC) algorithm is
used for selecting weighting matrices to overcome LQR design difficulties. The ABC algorithm is a swarm intelligence based optimization
algorithm and it can be used for multivariable function optimization efficiently. The simulation results justify that the ABC algorithm is a
very efficient way to determine LQR weighting matrices in comparison with trial and error method.

Keywords: ABC, LQR, Inverted Pendulum, Optimal Control, Weighting Matrices

quadratic optimal control problem. LQR has a simple process that

1. Introduction can achieve the closed loop linear quadratic optimal control with
linear state or output feedback [7], [8].
The inverted pendulum is an unstable and under-actuated system The most challenging part of LQR is selection of suitable
with highly nonlinear dynamics [1]. The control of inverted weighting matrices which affects the control input [9]. In general,
pendulum is a classic example for design, testing, and comparing selecting weighting matrices is performed by trial and error
of different control techniques as a consequence the control of
method, however, there does not exist a direct connection between
inverted pendulum has been a research interest in the field of
weighting matrices and time domain specifications such as
control engineering and the inverted pendulum has been a standard
overshoot percentage, settling time, and steady state error. There
tool in control laboratories for years. Another reason behind the are no relevant systematic techniques for selecting weighting
extensive studies of the inverted pendulum is that several important matrices, however; recently a few researchers have proposed
control systems can be modelled with the help of inverted artificial intelligence algorithms such as genetic algorithms and
pendulum [2]. Inverted pendulum reveals many interesting system-
particle swarm optimization algorithm for this goal [10], [11]. In
theoretic properties and its dynamics are fundamental to
addition, the Artificial Bee Colony algorithm, another swarm
maintenance balance, such as walking and two-wheeled robots [3],
intelligence optimization algorithm based on the intelligent
[4]. behaviour of honey bee swarm, can be used to determine LQR
Optimal control theory is a mathematical optimization method as weighting matrices.
an extension of the calculus variation and it has numerous The control problem is defined as selecting appropriate LQR
applications in control engineering. Determination of the control
weighting matrices to stabilize cart position and pole angle of
signals that will cause a process to meet the physical constraints
nonlinear inverted pendulum while minimizing settling time,
and also maximization or minimization of some performance
steady state error, and overshoot percentage. In this case study, the
criteria are the main objectives of optimal control theory [5]. A ABC algorithm is proposed to determine LQR weighting matrices
special case of the general nonlinear optimal control problem and simulation results illustrate that proposed method achieves
where the cost function is a quadratic function and the system desired control system characteristics and also has a satisfactory
dynamics are described by a set of linear differential equations is a
control performance.
linear quadratic optimal control problem. Linear quadratic optimal
The rest of this paper is organized as follows. Section II contains
control can be implemented in numerous control engineering
the nonlinear mathematical model of the inverted pendulum that is
problems, also it provides a basis for many other control techniques used in this study. In section III, the linear quadratic optimal
and hence it is very important for modern control theory [6]. Linear control problem based on linearized pendulum model is described.
Quadratic Regulator (LQR) is one of the main solutions for linear Section IV contains an overview of the ABC algorithm.
_______________________________________________________________________________________________________________________________________________________________
1
Department of Computer Engg. Cukurova University, Adana, Turkey Determination of LQR weighting matrices and simulation results
2
Department of Computer Engg. Cukurova University, Adana, Turkey are illustrated in section V followed by conclusion in section VI.
* Email: [email protected]
Note: This paper has been presented at the International Conference on
Advanced Technology&Sciences (ICAT'14) held in Antalya (Turkey), 2. The Inverted Pendulum
August 12-15, 2014.

This journal is © Advanced Technology & Science 2013 IJISAE, 2015, 3(1), 1–6 | 1
The inverted pendulum system used in this study consists of a
motor driven cart and a pendulum hinged to it, as shown in (Figure
1). The inverted pendulum system model used in this paper has
been suggested by Ogata [12]. The main aim of the controller is to
stabilize the pendulum as to keep pendulum upright position in
response to a change in cart position. Designed block diagram of
control system is shown in (Figure 2). Linear quadratic optimal
control where weighting matrices are selected by the ABC
algorithm can be used to determine design variables; integral gain
constant 𝐾𝑙 and feedback gain matrix 𝐾.
Assume that the rod is massless and the pendulum mass is
concentrated at the end of the rod. 𝜃 is the angel of the rod from
vertical line and the control force 𝑢 is applied to the cart. Also
assume that the sampling period 𝑇 is 0.1 𝑠, 𝑔 is 9.81 𝑚/𝑠 2 and
the following numerical values for 𝑀, 𝑚 and 𝑙:
Figure 1. Inverted pendulum system [12]
M  2kg, m  0.1kg , l  0.5m

In solving this design problem, we shall define state variables

𝑥1 , 𝑥2 , 𝑥3 and 𝑥4 as follows:

x1   , x2   , x3  x, x4  x

for nonlinear inverted pendulum model these variables can be

written as differential equations as follows: x1  x2 x1  x2

x1  x2 (1)
Figure 2. Block diagram of control system

u  cos x1  (M  m) g sin x1  ml cos x1 sin x1x22 where

x2  (2)
ml cos 2 x1  (M  m)l
 1.1048 0.1035 0 0
 2.1316 1.1048 0 0 
x3  x4 (3) G
 0.0025 0.0001 1 0.1
 
 0.0508 0.0025 0 0
u  ml sin x1x22  mg cos x1 sin x1
x4  (4)
M  m  m cos 2 x1
 0.0051
 0.1035
which are solved using fourth-order Runge-Kutta method in this H  
case study.  0.0025 
 
If displacement of the car is considered as the output of the system  0.0501 
then the output equation 𝑦 becomes
C  0 0 1 0
 x1 
x 
y   0 0 1 0  2  (5) D   0
 x3 
 
 x4 
3. Linear Quadratic Optimal Control
After linearizing the nonlinear differential (Equation 1) through The state space representation of a linear time-invariant (LTI)
(Equation 4) by taking 𝑠𝑖𝑛𝜃 ≑ 𝜃, 𝑐𝑜𝑠𝜃 ≑ 1, and 𝜃𝜃̇ 2 ≑ 0, the control system can be written as follows [12]:
discretized state and output equations of the system for linear
quadratic optimal control can be derived as follows: x(k  1)  Gx(k )  Hu(k ) (8)

x(k  1)  Gx(k )  Hu(k ) (6) y(k )  Cx(k ) (9)

y(k )  Cx(k )  Du(k ) (7) v(k )  v(k  1)  r (k )  y(k ) (10)

 x( k ) 
u   K ( k ) x(k )  K l v (k )    K Kl    (11)
 v(k ) 

where 𝑥(𝑘) is state vector (𝑛 vector) and 𝑢(𝑘) is control vector (𝑟

2 | IJISAE, 2015, 3(1), 1–6 This journal is © Advanced Technology & Science 2013
vector), respectively. 𝐺 and 𝐻 are 𝑛 × 𝑛 and 𝑛 × 𝑟 matrices, 𝑃 must be a positive definite, or for asymptotic stability a positive
indicate the constant system. 𝐾 is state feedback matrix. semi-definite.
At steady-state, the overall system dynamics with constant gain According to main design parameters of the linear quadratic
and integral feedback is described by the state equation which is a optimal control problem which are weighting matrices 𝑄 and 𝑅,
combination of (Equation 8) through ( Equation 11): the quality of the controller design depends on the choice of these
matrices. However, there are no relevant systematic techniques to
 x(k  1)   G 0  x(k)   H  0 select weighting matrices. This goal is performed by trial and error
v k  1     v k     u k     r k  (12)
   CG 1    CH  method in general. Although it depends on the designer’s
1 
experience, it is highly time consuming and selected values for
weighting matrices cannot establish a direct effect on the desired
In (Equation 12), it is assumed that the reference is a step change particular control system specifications.
hence 𝑟(𝑘) = 𝑟(𝑘 + 1). According to the importance of selecting weighting matrices 𝑄 and
Let us define 𝑅, selection of these matrices is performed by the ABC algorithm
in this paper.
 G 0
Gˆ    4. The Artificial Bee Colony Algorithm
 CG 1 
The Artificial Bee Colony (ABC) algorithm was introduced by
 H  Karaboga in 2005 as a new method which is based on the
Hˆ    intelligent behavior of honey bee swarms finding nectar and
 CH 
sharing the information of food resources with each other in the
field Swarm Intelligence to solve to optimize numeric benchmark
Kˆ   K  Kl  functions [13]. Then it was extended by Karaboga and Basturk and
presented to exceed other recognized heuristic methods like
 x Genetic Algorithm as well as Differential Evolution algorithm and
xˆ    Particle Swarm Optimization [14], [15]. The ABC algorithm has
v 
the advantages of strong robustness, fast convergence and high
flexibility, fewer control parameters and also it can be used for
0 
Dˆ    solving multidimensional and multimodal optimization problems
1  [16], [17].
In the ABC algorithm, the colony of artificial bees contains three
Therefore, the last equation can be written as follows: groups of bees: employed bees, onlooker bees and scout bees. An
employed bee memorizes the quality of the food source and finds

xˆ  k  1  Gˆ  HK 
ˆ ˆ xˆ  k   Dr
ˆ k  (13) a food source by modifying this information. Employed bees share
the food source information with other bees on the dance area.
The (Equation 13) is the state equation of the closed-loop control Onlooker bees watch the dance of employed bees within the hive
system. Its output equation is and find the food sources using the information provided by
employed bees. Scout bees search new food sources around the
ˆ ˆ(k )  [0]r (k )
y(k )  Cx (14) hive randomly. Both onlookers and scouts are also called
unemployed bees. The number of employed bees is equal to the
where 𝐶̂ = [𝐶 0]. number of food sources since each employed bee is associated with
Linear quadratic optimal control problem may be stated to find the one and only one food source. The position of a food source means
optimal input 𝑢 sequence that minimizes the quadratic a possible solution to the problem and the nectar amount of a food
performance index which is defined as: source corresponds to the fitness of the associated solution.
The general scheme of the ABC algorithm contains four phase;
1  * initialization phase, employed bees phase, onlooker bees phase,
J   x (k )Qx(k )  u* (k )Ru(k )
2 k o 
(15) and scout bees phase. Detailed pseudo code of the ABC algorithm
is as follows [18]:
1. Initialize the population of solutions
where Q is positive definite 𝑛 × 𝑛 matrix and 𝑅 is positive definite 2. Evaluate the population
𝑟 × 𝑟 matrix. The notation ( ∗ ) indicates complex-conjugate 3. cycle=1
transpose of a matrix. Matrices 𝑄 and 𝑅 are selected to weight the 4. repeat
relative importance of the performance measures caused by the 5. Produce new solutions (food source positions) vij in the
state vector and control vector, respectively. neighborhood of 𝑥𝑖𝑗 for the employed bees using the
The state feedback gain matrix is defined as follows: formula 𝑣𝑖𝑗 = 𝑥𝑖𝑗 + Φ𝑖𝑗 (𝑥𝑖𝑗 − 𝑥𝑘𝑗 )(k is a solution in the
neighborhood of 𝑖 ,Φ is a random number in the range
Kˆ  ( R  Hˆ *PHˆ )1 Hˆ *PGˆ (16) [-1,1] ) and evaluate them
6. Apply the greedy selection process between xi and 𝑣𝑖
7. Calculate the probability values 𝑃𝑖 for the solutions 𝑥𝑖 by
which is obtained by solving the following Ricatti equation: means of their fitness values using the following
equation
P  Q  Gˆ *PGˆ  Gˆ *PHˆ ( R  Hˆ *PHˆ )1 Hˆ *PGˆ (17)
fiti
Pi  SN
(18)
By the sense of Liapunov, for a stable matrix (𝐺̂ − 𝐻
̂𝐾̂ ), the matrix  fit
i 1
i

This journal is © Advanced Technology & Science 2013 IJISAE, 2015, 3(1), 1–6 | 3
 In order to calculate the fitness values of solutions we
employed the following equation

 1
 if fi  0
fiti  1  f i (19)
 1  abs  f  if f i  0
 i

Normalize 𝑃𝑖 values into [0, 1]

8. Produce the new solutions (new positions) vi for the
onlookers from the solutions xi , selected depending on Figure 3. Average fitness during the ABC convergence
𝑃𝑖 , and evaluate them
9. Apply the greedy selection process for the onlookers
where K1 , K 2 and K 3 are weight coefficients of the fitness
between xi and𝑣𝑖
10. Determine the abandoned solution (source), if exists, and functions and their values were chosen as 1.0.
replace it with a new randomly produced solution xi for The ABC algorithm was employed to select best Q and R
the scout using the following equation matrices that minimize f sum . The results of applying the ABC
algorithm to the problem are summarized as follows:
The parameters of the ABC algorithm are set in the range [0.1 100],
xij  minj  rand  0,1   max j  min j  (20) colony size=20 and max cycle=100. The average fitness values
during the ABC algorithm running is shown in (Figure 3). The
weighting matrices obtained by ABC algorithm are:
11. Memorize the best food source position (solution)
achieved so far
12. cycle=cycle+1 82.4186 34.8695 50.4278 17.4539 66.1159
13. until cycle= Maximum Cycle Number (MCN) 34.8695 24.7816 13.6630 3.5719 25.2813

Q  50.4278 13.6630 51.4173 10.8109 39.9392
5. Simulation Results  
17.4539 3.5719 10.8109 11.4281 18.6369 
The importance and difficulty of selecting weighting matrices was 66.1159 60.7356
 25.2813 39.9392 18.6369
mentioned above. The matrices Q and R were chosen by trial and
error method as follows in [12]:
R  0.1000
 10 0 0 0
0
 0 1 0 0 0  Using the Q and R matrices obtained by the ABC algorithm the
 matrix P is calculated by (Equation 17) as follows:
Q 0 0 100 0 0  , R  1
 
 0 0 0 1 0  17385 3737.4 15220 6395.5 2403.4
 0  3737.4 823.83 507.19
0 0 0 1   3315 1381.7
P   15220 3315 15439 5924.9 2660.2
 
Based on these matrices, feedback gain matrix K and integral gain  6395.5 1381.7 5924.9 2436.7 952.22
constant K l are determined as follows:  2403.4 507.19 2660.2 952.22 644.42 


K  64.9346 14.4819 10.8475 9.2871 Moreover, based on these matrices, feedback gain matrix 𝐾and
Kl  -0.5189
integral gain constant 𝐾𝑙 are determined as follows:

K  -137.7804 -31.1832 -68.8161 -35.4986

The goal of this simulation is to reduce the settling time  ts  of
unit-step response y (k) (the cart position) without an overshoot
Kl  -6.9144
 os  or with a minimum overshoot also minimize steady-state
error (ess ) . The objective weighting method where multiple
objective functions are combined into one objective function f sum Table 1. Performance Results
can be used for multi-objective optimization [19]. The objective
function defined as: Trial and Error [12] The Proposed

f sum  K1ts  K2 os  K3ess (21) ts  s  4.8987 1.4963

os  %  0 0.1038
004
ess 2.3938 10 5.3479 10009
f sum 4.8989 1.6002

4 | IJISAE, 2015, 3(1), 1–6 This journal is © Advanced Technology & Science 2013
 References
[1] K. J. Aström and K. Furuta, "Swinging up a pendulum by
energy control," Automatica, vol. 36, no. 2, pp. 287-295,
2000.
[2] J. Anderson, "Learning to control of an inverted pendulum
using neural networks," IEEE Control Systems Magazine,
vol. 9, no. 3, pp. 31-37, 1989.
[3] A. Kuo, "The six determinants of gaint and the inverted
pendulum analogy: Adynamic walking perspective," Human
Movement, vol. 26, pp. 617-656, 2007.
[4] S. Jeong and T. Takahashi, "Wheeled inverted pendulum type
Figure 4. Cart Position and Pendulum Angle for Trial and Error Method asistant robot: inverted mobile, standing and sitting motions,"
in IEEE/RSJ International Conferance on Intelligent Robots
and Systems, 2007, 2007.
[5] D. E. Kirk, Optimal control theory: an introduction, New
Jersey: Prentice-Hall, Inc., 1970.
[6] M. Athans, "The status of optimal control theory and
applications for for deterministic systems," IEEE
Transactions on Automatic Control, vol. 11, no. 3, pp. 580-
596, 1966.
[7] R. E. Kalman, "When is a linear control system optimal?,"
Journal of Basic Engineering, vol. 86, pp. 51-56, 1964.
[8] H. Kwakernaak and R. Sivan, Linear optimal control systems,
New York: Wiley-Interscience, 1972.
[9] J. V. D. F. Neto, I. S. Abreu and F. N. Da Silva, "Neural-
genetic synthesis for state-space controllers based on linear
quadratic regulator design for eigenstructure assignment,"
Figure 5. Cart Position and Pendulum Angle for Proposed Method
Trans. Sys. Man Cyber. Part B, vol. 40, no. 2, pp. 266-285,
2010.
Since the matrix P is positive definite, the closed-loop control [10] C. P. Bottura, J. V. da Fonseca Neto, "Parallel eigenstructure

system is stable. That is, all eigenvalues of Gˆ  HK 
ˆ ˆ lie inside assignment via LQR design and genetic algorithms," in
of the unit circle in the following: Proceedings of the American Control Conference 1999. Vol.
4. , San Diago, 1999.
z1  0.2077 [11] S. Mobayen, A. Rabiei, M. Moradi and B. Mohammady,
"Linear quadratic optimal control system design using
z2  0.77628 + 0.28782i
particle swarm optimization algorithm," International Journal
z3  0.77628 - 0.28782i of the Physical Sciences, vol. 6(30), pp. 6958-6966, 2011.
z4  0.75904 [12] K. Ogata, "Quadratic Optimal Control Systems," in Discrete-
Time Control Systems 2nd Edition, Englewood Cliffs, NJ,
z5  0.72792 Prentice-Hall, 1995, pp. 566-633.
[13] D. Karaboga, "An idea based on honey bee swarm for
The performance results are presented in (Table 1). Also plots of numerical optimization," Erciyes University, Kayseri,
position of the cart and pendulum angels as unit-step response of Turkey, 2005.
designed system has been is in (Figure 4) and (Figure 5) for both
control systems. Cart position is indicated by solid line and [14] D. Karaboga and B. Basturk, "An artificial bee colony (ABC)
pendulum angle is indicated by dotted line in these figures. Both algorithm for numeric function optimization.," in IEEE
simulations are performed on the nonlinear pendulum model given Symp. Swarm Intelligence, Indianapolis, 2006.
by (Equation 1) through (Equation 5). [15] D. Karaboga and B. Basturk, "On the performance of artificial
bee colony (ABC) algorithm," Applied Soft Computing, vol.
6. Conclusion 8, no. 1, pp. 687-697, 2008.
[16] D. Karaboga and B. Basturk, "A powerful and efficient
In this paper, the ABC algorithm based linear optimal controller
algorithm for numerical function optimization: artificial bee
design for nonlinear inverted pendulum has been presented. The
colony (ABC) algorithm," Journal of Global Optimization,
ABC algorithm has been employed to determine linear quadratic
vol. 39, no. 3, pp. 459-471, 2007.
optimal controller weighting matrices. Using the ABC algorithm
has been proved to be effective and feasible to select weighting [17] D. Karaboga and B. Akay, "A comparative study of Artificial
matrices for nonlinear pendulum controller design more than trial Bee Colony algorithm," Applied Mathematics and
and error method. Also it has been shown that it can optimize Computation, vol. 214, no. 1, p. 108–132, 2009
multiple time domain control system specifications such as settling
time, overshot and steady state error.

This journal is © Advanced Technology & Science 2013 IJISAE, 2015, 3(1), 1–6 | 5
[18] "Detailed Pseudocode of the ABC Algorithm," 14 October [19] R. Coban , "A fuzzy controller design for nuclear research
2008. [Online]. Available: reactors using the particle swarm optimization algorithm,"
http://mf.erciyes.edu.tr/abc/pub/PsuedoCode.pdf. [Accessed Nuclear Engineering and Design, vol. 241, no. 5, pp. 1899-
1 March 2014]. 1908, 2011.

6 | IJISAE, 2015, 3(1), 1–6 This journal is © Advanced Technology & Science 2013