LQ R Inverted Pendulum
LQ R Inverted Pendulum
LQ R Inverted Pendulum
AAE 364L
This experiment is devoted to the inverted pendulum. Clearly, the inverted pendulum
will fall without any control. We will design a controller to balance the pendulum upright.
1
are given by
(Mc + Mp )Ip + Mc Mp lp2 + Mp2 lp2 sin(α)2 ẍc + Ip + Mp lp2 Beq ẋc
= −Mp lp Bp cos(α)α̇ − Mp2 lp3 + Ip Mp lp sin(α)α̇2 + Ip + Mp lp2 Fc + Mp2 lp2 g cos(α) sin(α);
= (Mc + Mp ) Mp glp sin(α) − Mp lp cos(α)Beq ẋc − Mp2 lp2 sin(α) cos(α)α̇2 + Mp lp cos(α)Fc ;
ηg Kg ηm Kt (vrmp − Kg Km ẋc )
Fc = 2
. (0.1)
Rm rmp
Here Fc is the force on the cart. The notation and values are given in the table in the
Appendix. These values are given also given in the MATLAB file “setup lab ip01 2 sip.m”
posted on the Web page for the course. So you do not have to calculate any of these values
for the Lab. The linearized equations of motion about α = 0 are determined by
Mp2 lp2 gα − (Ip + Mp lp2 )Beq ẋc − Mp lp Bp α̇ + (Ip + Mp lp2 )Fc
ẍc =
(Mc + Mp )Ip + Mc Mp lp2
(Mc + Mp )Mp glp α − (Mc + Mp )Bp α̇ − Mp lp Beq ẋc + Mp lp Fc
α̈ =
(Mc + Mp )Ip + Mc Mp lp2
ηg Kg ηm Kt (vrmp − Kg Km ẋc )
Fc = 2
. (0.2)
Rm rmp
Now let us convert the linear equations to a state space model of the form
ẋ = Ax + Bv.
Here A is a 4 × 4 matrix, B is a column vector of length 4 and the input v is the voltage
to the motor. To convert the system in (0.2) to state space we will use the following state
variables:
x1 xc
x2 α
x= x3 = ẋc .
(0.3)
x4 α̇
By consulting the values in the table in the Appendix, we computed A and B, that is,
0 0 1 0 0
0 0 0 1 0
A= 0 1.5216 −11.6513 −0.0049
and B =
1.5304 .
(0.4)
0 26.1093 −26.8458 −0.0841 3.5261
It is emphasized that this A and B in (0.4) is computed for the IP02 cart with the 0.37 kg
weight attached and the long pendulum. The matrices A and B will change if we take off
the weight or switch to the medium pendulum. The matrices A and B in (0.4) are contained
2
in the MATLAB setup file “setup lab ip01 2 sip.m” posted on the course Web page. So
you do not have to retype A and B in MATLAB. Because the inverted pendulum is unstable
and the state equation ẋ = Ax + Bv is the linear approximation for the inverted pendulum,
the matrix A is unstable. In fact, the eigenvalues for A are given by
eig(A) = {0, 4.8231, −12.0133, −4.5452}. (0.5)
As expected, A is unstable.
Let us compute the transfer function from the voltage v to the output xc . Since x1 = xc ,
the corresponding output matrix C is given by
C= 1 0 0 0 . (0.6)
By using ss2tf in MATLAB, we see that the transfer function from the voltage v to the
position xc of the cart is given by
Xc (s) 1.53s2 + 0.1114s − 34.59
= C(sI − A)−1 B = 4 . (0.7)
V (s) s + 11.74s3 − 25.26s2 − 263.4s
Notice that this is a fourth order system. Because the denominator contains terms with
(s)
negative coefficients, it follows that the transfer function XVc(s) is unstable. In fact, the
Xc (s) Xc (s)
poles of V (s)
are the eigenvalues of A. In other words, the poles of V (s)
are given by
Xc (s)
eig(A) = {0, 4.8231, −12.0133, −4.5452}. Therefore is unstable.
V (s)
Now let us compute the transfer function from the voltage v to the angle α. Since x2 = α,
the corresponding output matrix C is given by
C= 0 1 0 0 . (0.8)
By using ss2tf in MATLAB, we see that the transfer function from the voltage v to the
angle α is determined by
α(s) 3.526s
= C(sI − A)−1 B = 3 2
. (0.9)
V (s) s + 11.74s − 25.26s − 263.4
Finally, it is noted that this is an unstable third order system.
The displacement of the end of the pendulum along the track is given by xe = xc +
Lp sin(α) where Lp is the length of the pendulum; see Figure 1. (Notice that lp is the
distance to the center of mass of the pendulum and Lp is the distance to the end of the
pendulum.) For small angles xe ≈ xc + Lp α. Since x1 = xc , x2 = α and Lp = 0.6413 m, the
output matrix C corresponding to xe is given by
C = 1 0.6413 0 0 . (0.10)
By using ss2tf in MATLAB we see that the transfer function from the voltage v to the end
of the pendulum xe is determined by
Xe (s) 3.792s2 + 0.1114s − 34.59
= C(sI − A)−1 B = 4 . (0.11)
V (s) s + 11.74s3 − 25.26s2 − 263.4s
Finally, it is noted that this is an unstable fourth order system.
3
1 The linear quadratic regulator
You can use pole placement to design a feedback controller for the inverted pendulum. One
disadvantage of the pole placement procedure is that placing the poles at a desired location
can lead to large gains. In this section, we will present the linear quadratic regulator (LQR)
problem, and show how one can use the LQR for control design. It turns out that in many
instances, the LQR method is superior to the pole placement method. Finally, you can
choose either pole placement or LQR for your control design of the inverted pendulum. In
fact, you can use any control method that you want as long as it works. However, we believe
you will discover that the LQR method will work very nicely for the inverted pendulum.
Now let us present the linear quadratic regulator (LQR) method. The proof behind the
LQR is given in A&AE 564. Here we will just use MATLAB to design a LQR controller. To
set up the LQR problem, consider a controllable state space system of the form
ẋ = Ax + Bv. (1.1)
Here A is a n × n matrix and B is a column vector of length n and the state
x1
x2
x= ...
xn
is a vector of length n. Let R > 0 and Q be a diagonal matrix of the form
q1 0 · · · 0
0 q2 · · · 0
Q= ... ... . . . ... ,
(1.2)
0 0 ··· qn
where qj ≥ 0 for all j. Now consider the optimization problem
Z ∞ Z ∞ X n
!
x∗ Qx + Rv 2 dt = min qj x2j + Rv 2 dt
min
0 0 j=1
subject to ẋ = Ax + Bv. (1.3)
(The conjugate transpose of a vector z is denoted by z ∗ .) The control engineer chooses the
weights {qj }n1 and R > 0. It is emphasized that the weight R must be strictly positive. The
optimal control v or solution to this optimization problem is unique and given by v = −Kx
where K is a state gain matrix, that is, K is a row matrix of length n. The MATLAB
command to compute K is given by
K = lqr(A, B, Q, R). (1.4)
In this case, the closed loop system corresponding to the optimal control v = −Kx is given
by
ẋ = (A − BK)x.
4
Finally, it is noted that A − BK is stable.
The state weights {qj }n1 are chosen to emphasize the response of certain states, while the
control weight R is chosen to select how much control effort v is used to solve the optimization
problem in (1.3). For example, if q1 and q3 are large, then the weights q1 and q3 are placing
a large penalty on the states x1 and x3 , that is, if x1 or x3 is large, then the large weights q1
and q3 will amplify the effect of x1 or x in the optimization problem. Since the optimization
R ∞ P3n 2 2
problem is trying to minimize 0 j=0 qj xj + Rv dt, the optimal control v must force
the states x1 and x3 to be small. So if q1 and q3 are large, then the control designer is trying
to make the states x1 and x3 small and places less emphasis on how the other states respond.
If the control weight R is large relative to {qj }n1 , then the weight R is placing a large
penalty on the control effort v, that is, if the control v is large, then the large weight
R relative to {qj }n1 will amplify the effect of the control v in the optimization problem.
R ∞ Pn 2 2
Since the optimization problem is trying to minimize 0 j=0 qj xj + Rv dt, the optimal
control v must be small in order to solve the optimization problem in (1.3). In other words,
if R is large relative to {qj }n1 , then control effort is expensive. So for large R, the gain K
should be small and the response might be slow. On the other hand, if R is small relative
to max{qj }n1 , then there is virtually no penalty on the control effort v in the minimization
problem (1.3), and the optimal control v can be large. In other words, if R is small relative to
max{qj }n1 , then control is cheap. So for small R, the gain K might be large and the response
should be faster. Finally, it is noted that a large gain may cause problems. Recall that our
system has saturation. So a large gain may lead to instability. Furthermore, a large gain has
a faster response. This can lead to chatter in the system, that is, the system moves so fast
that the gears chattering back and forth even when the system is trying to rest. Summing
up this analysis we obtain:
• If qj is large, then the control will try to make the state xj small relative to the other
states.
• If qj is small, then the control will not place much emphasis on xj and xj could be large
relative to the other states.
• If R is large relative to {qj }n1 , then control is expensive and the response may be slow.
• If R is small relative to max{qj }n1 , then control is cheap and the response should be
faster.
Now consider our state space for in the inverted pendulum in (0.4). The manufacturer
suggests starting with R = 3 × 10−4 and
0.75 0 0 0 q1 0 0 0
0 4 0 0 0 q2 0 0
Q= 0 0 0 0 = 0 0 q3 0 .
(1.5)
0 0 0 0 0 0 0 q4
Recall that x1 = xc is the position of the cart, x2 = α is the angle of the pendulum from
the vertical position, while x3 = ẋc and x4 = α̇. Since R is small the control engineer is
5
allowing the optimal control to use a large control effort. So the control should be fast. The
0.75 weight on x1 = xc says that the control engineer wants to keep the position of the cart
from moving too much. Notice that 4 is the largest element in Q. By choosing the weight 4
on x2 = α the designer is placing a larger emphasis on the angle α, over any other state. In
other words, the designer is saying that making α small is more important than making any
other state small. Since q3 = 0 and q4 = 0, this design is not trying to place any particular
emphasis on the velocity ẋc of the cart or the angular velocity α̇ of the pendulum. Finally,
it is noted that the gain K corresponding to Q in (1.5) and R = 3 × 10−4 is given by
3
3
−50 180.16 −50.26 28.49 = lqr(A, B, diag( 4
4 0 0 ), 4 ). (1.6)
10
There are all kinds of LQR controllers which will work. For example, R = 1 and
20 0 0 0
0 20 0 0
Q= 0 0
. (1.7)
10 0
0 0 0 3
In practice, the designer will simulate and test many different R and Q matrices in the LQR
design, before arriving at the final controller.
ẋ = Ax + Bv. (2.1)
The input is the voltage v and the vector x is the state; see (0.3). Here A is the 4 × 4
matrix, B is a column vector of length 4, determined by the cart and pendulum; see (0.4).
Your problem is to design a state feedback controller K = K(1) K(2) K(3) K(4) which
makes the pendulum stand up. It is very important to note that in practice, you can use
any control method that you like as long as it works. The idea here is to introduce you to
the LQR method to design a state feedback gain K.
To begin your control design, go to the course Web page and load the following MATLAB
files in your computer
• setup lab ip01 2 sip.m
• setup ip01 2 configuration.m
6
• setup sp configuration.m
• SSIP ABCD eqns.m
• d gui lqr tuning.m
• d ip01 2 sip lqr.m
• s sip lqr.mdl
Run the MATLAB file “setup lab ip01 2 sip.m” and open the Simulink file “s sip lqr.mdl”.
This Simulink file is displayed in Figure 2. You can use this Simulink file to simulate your
controller design. In the Simulink file set the amplitude of the position set point equal to
zero; see Figure 2. Check to make sure that the setup program “setup lab ip01 2 sip.m”
has specified the following variables in MATLAB:
(i) CART TYPE = ‘IP02’
(ii) IP02 LOAD TYPE = ‘WEIGHT’
(iii) PEND TYPE = ‘LONG 24IN’
(iv) IC ALPHA0 = 0.2.
This means that we are using the manufacturer’s cart IP02, the long pendulum and we have
added the 0.37 kg weight on the cart, and initial condition α(0) = 12 degrees (about 0.2 radi-
ans). If (i) to (iii) does not hold go into the “setup lab ip01 2 sip.m” file and add or take
off the comment % on the appropriate lines. Then run the file “setup lab ip01 2 sip.m”
again. If IC ALPHA0 6= 0.2, then simply set IC ALPHA0 = 0.2 in MATLAB before you
run the Simulink file. All the variables that you need to run the simulation are now loaded
in the computer. MATLAB contains all the values in the Table in the Appendix.
Now using the pole placement method, find a state feedback gain K to balance the
pendulum upright. Recall that the command used in MATLAB is
K = place(A, B, λ1 λ2 λ3 λ4 ). (2.2)
Recall that to obtain a stable system, the real parts of {λj }41 must all be negative. Moreover,
since the system is real, if we choose any of the poles to be complex then its complex conjugate
must also be chosen. In our case, you may choose all poles to be negative real, or λ1 , λ2
are negative real and λ3 is complex with negative real part with λ4 = λ̄3 , or λ1 , λ3 complex
with negative real parts whereas λ2 = λ̄1 , λ4 = λ̄3 . Finally, recall that the complex part
of any pole corresponds to the sinusoidal frequency of the response. Note that the natural
frequency of the pendulum is about 5 rad/s. So we may want some poles with frequencies
much faster than 5 rad/s.
Now using the LQR method, design a state feedback gain K to balance the pendulum
upright. The LQR command in MATLAB is
K = lqr(A, B, diag( q1 q2 q3 q4 ), R). (2.3)
7
The poles of this feedback system are the eigenvalues of A − BK.
You can test the state gain K you obtain from both methods by typing K in MATLAB
and then running the Simulink file “s sip lqr.mdl”. This Simulink file will see if your
controllers can balance the pendulum upright when the pendulum is initially leaning 0.2
radians off center. Bring your best state feedback gains K you obtain from both design
methods along with the corresponding plots of the angle and position to the Lab with you.
xc_des
Control Effort:
Vm (V) X_eom
Position Setpoint (m) xc (m) 4
alpha (rd)
u = Vm 4
20e−3 K(1) Vm (V) Scopes
xc_dot (m/s)
Position UPM Voltage
Setpoint Setpoint Saturation alpha_dot (rd/s)
Amplitude (m)
SIP + IP01_2:
Non−Linear EOM
4
X = [ xc; alpha; xc_dot; alpha_dot ]
K* u
LQR Gain
2.1 Pre-Lab to balance the pendulum. Due at the beginning of the lab experi-
ment. You will not be allowed run the lab experiment with out a complete
pre-lab.
(i) Hand in the values of the feedback gains K that you obtained using Simulink to balance
the inverted pendulum. You will be given no credit for using the state feedback gain
K in (1.6) or (1.8).
(ii) Hand in the plots of the poles of the feedback system with pole placement and LQR
method. Put them on separate plots, but with the same scale, same range, same
position of origin.
(iii) Hand in the plots for the angle α for the both gains K that you used in Simulink on
the same graph. Hand in the plots for the position xc for the both gains K that you
used in Simulink on the same graph.
8
“setup lab ip01 2 sip.m”. In MATLAB command window, type X M AX = 0.6;
X M IN = −0.6;.
(ii) Enter your best gain K you obtain from the pole placement method in MATLAB, that
is,
K = K(1) K(2) K(3) K(4) .
9
2.3 In your lab report include the following under Part (i):
(a) Hand in the plots of the poles of the feedback system with pole placement and LQR
method. Put them on separate plots, but with the same scale, same range, same
position of origin.
(b) Clearly state the values of these poles in a Table, NOT in text.
(c) Clearly state the Matrix Q and R you choose for your LQR design in equation, NOT
in text.
• Initial condition xc = 0, α = 0.2, ẋc = 0, α̇ = 0
(d) Hand in the plots of position you obtain from the simulation using pole placement
and LQR, from the experiment using pole placement, and LQR on the SAME graph
(you should have four lines). Make sure the initial condition xc here is zero. If your
experimental results do not have zero initial xc , scale them to zero (Subtract nonzero
initial condition off).
(e) Hand in the plots of angle you obtain from the simulation using pole placement and
LQR, from the experiment using pole placement, and LQR on the SAME graph (you
should have four lines). Your initial condition from the simulation should be 0.2 rad.
However, the initial condition from the experiment will not be exactly 0.2 rad.
• Initial condition xc = 0, α = 0, ẋc = 0, α̇ = α̇
For the following parts, the size of α̇ can not be measured and must be determined
from the experiment data. So use the simulation to determined the approximate value
of α̇. This can be done by first changing the parameter IC ALP HA0 to zero. Then
in the simulink model, go into the block “SIP+IP01 2: Non-linear EOM”, then go
into the block “xc ddot EOM”, then go into the block “alpha ddot EOM”, finally
go into the integrator block “1/s” behind the words “alpha ddot” and change the
“initial condition”. Simulate various α̇ so that your simulation result resembles the
experimental result.
(f) Hand in the plots of position you obtain from the simulation using pole placement,
from the experiment using pole placement, on the SAME graph (you should have two
lines). Make sure the initial condition here is zero. If your experimental results do not
have zero initial condition, scale them to zero (Subtract nonzero initial condition off).
Do the same with the simulation and experiment results using LQR.
(g) Hand in the plots of angle you obtain from the simulation using pole placement, from
the experiment using pole placement on the SAME graph (you should have two lines).
Your initial condition from both the simulation and the experiment should be 0 rad.
Do the same with the simulation and experiment results using LQR.
10
3 Part (ii): Moving the balanced pendulum
In this part we will use a state space integral controller to move the inverted pendulum
from one point on the track to another without letting the pendulum fall. In this section
we will be using the cart IP02 with the 0.37 kg weight and the long pendulum. For these
specifications run the MATLAB file “setup lab ip02 spg.m”. Make sure that the following
parameters are specified in MATLAB:
(i) CART TYPE = ‘IP02’
(ii) IP02 LOAD TYPE = ‘WEIGHT’
(iii) PEND TYPE = ‘LONG 24IN’
(iv) IC ALPHA0 = 0.
Finally, it is noted that in this part we have set IC ALPHA0 = 0.
The idea behind our design is to incorporate an integral controller in our LQR design.
(You can use pole placement. However, LQR will be more effective.) To this end, consider
the four dimensional state variable system in (0.4). Now let us define a new state variable
Z t
x5 = (u(σ) − x1 (σ)) dσ or equivalently ẋ5 = u − x1 . (3.1)
0
Here u is the new input. Recall that x1 is the position of the cart. In our problem u = u0
will be a constant. So x5 will integrate u0 − xc . Using ẋ5 = u − x1 , the state variable system
in (0.4) now becomes
ẋ = Ai x + Bi v + Du. (3.2)
Recall that the new state matrix Ai and Bi are given by
0 b1
0 b2
A 0 and Bi = B = b3
Ai = 0
0 b4
−1 0 0 0 0 0
xc 0
α
0
x = ẋ c
and D = 0 .
R α̇ 0
(u − xc )dt 1
Using the matrices A and B in (0.4) one can easily construct Ai , Bi and D in MATLAB.
Recall that you already have A and B in MATLAB from your setup file. In MATLAB the
matrices Ai and Bi are computed by
Ai = [A, zeros(4, 1); −1, 0, 0, 0, 0]
Bi = [B; 0] .
11
Finally, it is noted that Ai is a 5 × 5 matrix and Bi is a column vector of length 5.
It turns out that the pair {Ai , Bi } is controllable. So one can use the LQR (or place)
command in MATLAB to design a state feedback controller K. Using v = −Kx in the new
state variable system in (3.2), we arrive at
Now choose u to be the step u(t) = u0 for all t ≥ 0. Because all the eigenvalues of Ai − Bi K
are in the open left hand plane, the state space system in (3.3) will move to steady state,
that is, ẋ = 0 in steady state. In particular, 0 = ẋ5 = u0 − x1 . In other words, in steady
state xc = x1 = u0 , and the cart will move to the position u0 on the track.
Now we would like to design a controller to balance the pendulum and move the cart to
any position from −0.2 to 0.2 meters on the track. To accomplish this choose the weights
{qi }51 and R that you think will work. Notice that in this case, there are five state weights
{qi }51 . The weights q1 , q2 , q3 , q4 and q5 respectively correspond to the position xc of the
cart, the angle α of the pendulum,
R the velocity ẋc of the cart, the angular velocity α̇ of the
pendulum, and the integral (u0 − xc )dt. Then in MATLAB type
K = lqr(Ai , Bi , diag( q1 q2 q3 q4 q5 ), R). (3.4)
To test your control design, you can use the Simulink file “aae364pinv2.mdl” on the web
sight and displayed in Figure 3, or simply rebuild the Simulink file “s sip lqr.mdl” to the
one specified in Figure 3. To test your design set the step block to 1 and slider gain to any
specified position u0 in [−0.2, 0.2]. Then choose the weights Q and R that you think will
work and run the LQR command in MATLAB. Finally, run the Simulink file in Figure 3
using various positions on the slider gain. Keep changing the weights Q and R until you find
the state feedback gain which achieves your desired response. Now set the slider gain to 0.2
and in the step block change step time to 5, initial value to -1, and final value to 1. Simulate
the system and keep the result. Bring your best gain matrix K and this simulation result to
the Lab.
position
LQR Gain
12
3.1 Pre-Lab for the integral controller. Due at the beginning of the lab ex-
periment. You will not be allowed to run the lab experiment with out a
complete pre-lab.
(i) Hand in your best values for the state gain K that you computed from Simulink.
(ii) Hand in the plots of position and angle from the simulation.
(iv) In the same directory, open up labpinv2 in Simulink and hit build.
(v) In the WinCon server window, open the plots for the position and angle, under the tab
update ⇒ buffer; set to 30 seconds.
(vi) Put the cart in the middle of the track, with the pendulum in the gantry or downward
position. The computer thinks that this position is the angle π. So make sure that the
pendulum is not swinging when you hit the start button.
(vii) Hit the start button in WinCon and wait until this button displays stop in red. Now
slowly move the pendulum to the upright position. As soon as the angle hits zero, the
controller will start or kick in. In case your controller does not work, be prepared to
hit stop. In other words, if the pendulum falls or the cart runs off the track hit stop.
(viii) Move the slider gain to the position u0 that the TA tells you to. The cart and the
pendulum should move to the position u0 on the track. The pendulum should remain
in the upright position as the cart is moving.
(ix) Use the slider gain to move the cart to the zero position. Wait until the system
stabilized. Then change the slider gain A to -0.2, that is, you try move the cart to
the position xc = −0.2 meters. Wait for 5 seconds, then immediately change the slider
gain A to 0.2. The cart should move to the position xc = 0.2 meters. Wait until the
system stabilized, then hit stop.
(x) Save the position and angle in MATLAB, that is, go to: File ⇒ Save ⇒ save as Mat
file. Take this Mat file with you.
13
3.3 In your lab report include the following under Part (iii):
(a) Hand in the plots of the poles of the feedback system.
(b) Clearly state the values of these poles in a Table, NOT in text.
(c) Clearly state the Matrix Q and R you choose for your LQR design in equation, NOT
in text.
(d) Hand in the plots of position you obtain from the simulation and the experiment on
the SAME graph.
(e) Hand in the plots of angle you obtain from the simulation and the experiment on the
SAME graph.
14