EE 622 Optimal Control Systems: Assignment 4

Instructor: Dwaipayan Mukherjee∗

Indian Institute of Technology Bombay, Mumbai- 400076, India

1. Consider the following dynamical system:

     
0 1 0 1
ẋ = x(t) + u(t); x(0) =
0 1 1 2

with the cost function

Z ∞
x(t)T x(t) + u2 (t) dt


J=
0

Find the linear state-feedback control law that minimizes J. Also, find the value of the performance index for
the closed-loop system.
2. Using the HJB-equation, and a candidate quadratic function V (t, x), find the optimal control action that
minimizes this performance index:
Z ∞
J = (x(1))2 + (x2 (t)u2 (t))dt
0

where the dynamics of the system is given by:

ẋ(t) = x(t)u(t), x(0) = 1

3. Find the shortest path to node 6 from all other nodes using Dynamic Programming (refer to fig.1)

Figure 1: Q.3 Find shortest path to node 6

4. Find the shortest path from node 1 to all the other nodes using Dynamic Programming (refer to fig.2)
5. The first order linear system

ẋ = −10x + u

is to be controlled to minimize the performance measure

Z 0.04
1 2 1 2 1 2
J = x (0.04) + x + u dt.
2 0 4 2
The admissible state and control values are not constrained by any boundaries. Find the optimal control law
by using the Hamilton-Jacobi-Bellman equation.
∗ Asst. Professor, Electrical Engineering, Office: EE 214D, e-mail: [email protected]

1
 Figure 2: Q.4 Find shortest path from node 1

6. The first-order discrete system

x(k + 1) = 0.5x(k) + u(k)
is to be transferred to the origin in two stages (x(2) = 0) while the performance measure
1
X
J= [|x(k)| + 5|u(k)|]
k=0

is minimized.
(a) Use the method of dynamic programming to determine the optimal control law for each of the heavily
dotted points in Fig 3. Assume that the admissible control values are quantized into the levels 1, 0.5, 0,
-0.5, -1.

Figure 3

(b) Find the optimal control sequence {u∗ (0), u∗ (1)} that corresponds to the initial state x(0) = −2.
7. Consider the linear regulator problem, with the system described by the linear state equations
ẋ(t) = A(t)x(t) + B(t)u(t)
and the performance measure to be minimized is
Z tf
1 1
J = xT (tf )Hx(tf ) + xT (t)Q(t)x(t) + uT (t)Ru(t) dt
2 2 t0

where H and Q are real symmetric positive semi-definite matrices and R is a real positive definite matrix.
Show that if tf is fixed and x(tf ) = 0 the optimal control law is
u∗ (t) = R−1 B T (t)[ϕ12 (tf , t)]−1 ϕ11 (tf , t)x(t)

8. If the conditions for a time invariant optimal control law are satisfied by the linear regulator problem described
above, The constant matrix K must be the solution of the non-linear algebraic equations
0 = −KA − AT K − Q + KBR−1 B T K.
Using this result, determine the optimal control laws for:

2
 (a) The first order system ẋ(t) = ax(t) + u(t) with performance measure
Z ∞
J= [qx2 (t) + ru2 (t)] dt q, r > 0
0
q
Show the variation of the pole of the closed loop system for 0 < r < inf.
(b) The system

ẋ1 (t) = x2 (t)

ẋ2 (t) = −4x1 (t) − 4x2 (t) + u(t)

and the performance measure

Z ∞
J= [20x21 (t) + 5x22 (t) + u2 (t)] dt.
0

Find the location of the poles of the controlled (closed-loop) system and compare with the pole locations
for the open-loop system.
9. The approximating difference equation representation for a continuously operating system is

x(k + 1) = 0.75x(k) + u(k)

It is desired to bring the system state to the target set S defined by 0 ≤ x(2) ≤ 2 with minimum expenditure
of control effort; i.e., minimize
J = u2 (0) + u2 (1).
The allowable state and control values are constrained by

0 ≤x(k) ≤ 6
−1 ≤u(k) ≤ 1.

Quantize the state values into the levels x(k) = 0, 2, 4, 6 for k = 0, 1, 2 and the control values into the levels
u(k) = −1, −0.5, 0, 0.5, 1 for k = 0, 1.
(a) Find the optimal control value(s) and minimum cost for each point on the state grid. Use linear interpo-
lation.
(b) What is the optimal control sequence {u∗ (0), u∗ (1)} if x(0) = 6?
10. Consider the system
ẋ(t) = a(x(t), u(t), t)
which is to be controlled to minimize some performance measure J. The admissible state and control values
are bounded, and, in addition, the control must satisfy the total energy constraint
Z tf
u2 (t)dt ≤ M ;
t0

M is a specified positive number. Can this problem be solved by applying dynamic programming? Explain.
11. Let W be the total available resources. The problem is to ascertain the portion of the available resources
to allocate to each of N activities in order to maximize the total return. Let wi be the quantity of available
∗
resources allocated to activity i and vi be per unit values of carrying out activity i. Let JN (W ) be the maximum
return that can be obtained by allocating resources of amount W among N activities, clearly,
(N )
X
∗
JN (W ) = max wi vi (1)
w1 ,...,wN ≥0
w1 +...+wN ≤W i=1

(a) Show that equation (1) leads to the functional recurrence equation
∗ ∗

JN (W ) = max wN vN + JN −1 (W − wN )
wN
0≤wN ≤W

(b) Use part 11a to solve the following shipping problem: A truck of capacity 11,000Kg is to transport
automobiles, refrigerators, and kitchen sinks between point X to point Y . The item to be transported
have the weights and values shown in table below:
Description Wt/unit value/unit Value/wt
Automobile 4000 kg |3000 |0.75
Refrigerator 400kg |280 |0.70
Kitchen sink 100kg |50 |0.50

3
 Determine the number of each item that should be transported to maximize the total value of a shipment.
(c) Suppose that in shipping problem the value of second car is |2500, and each refrigerator after the tenth
is valued at only |250. The kitchen sinks remain at |50 a piece. Use dynamic programming to determine
the optimal loading schedule.
12. (a) Determine the optimal control law for the system

ẋ(t) = −x(t) + u(t)

to be transferred to the origin from an arbitrary initial state. The performance measure is
Z 1
1 2
J= [3x (t) + u2 (t)]dt
0 2

The admissible controls are not bounded.

(b) Determine the optimal control law for part 12a with x(1) free.
13. Find u∗ (t) as a function of x(t), t, and J ∗ for system;

x˙1 (t) = x2 (t)

x˙2 (t) = −x1 (t) + x2 (t) + u(t)

with the performance measure;
Z T
J= 0.5[q1 x21 + q2 x22 + u2 ]dt
0

Here, q1 , q2 > 0 and the control is constrained as: |u(t)| ≤ 1

14. Determine the discrete approximations for the following system;

x˙1 (t) = x2 (t)

x˙2 (t) = −x1 (t) + (1 − x21 (t))x2 (t) + u(t)

which is to be controlled to minimize the performance measure:
Z T
2
J = (x1 (T ) − 5) + [x22 + 20(x1 − 5)2 + u2 ]dt
0

Here, T = 10 and interval △t = 0.01

15. The second-order plant equations are given by:

ẋ(t) = x2 (t), x1 (0) = 2

x2 (t) = −2x1 (t) + x2 (t) + u(t), x2 (0) = −3

and the performance index is :

Z ∞
J = 0.5 [2x21 + 6x1 x2 + 5x22 + 0.25u2 ]dt
0

Obtain the feedback optimal control law.

16. Consider a first-order system:
ẋ(t) = −3x(t) + u(t)
and the cost function (CF) as: Z ∞
J= (x2 (t) + u2 (t))dt
0

where x(0) = 1 and the final state x(∞) = 0. Find the open-loop and closed-loop optimal controllers.
17. Obtain the control law for following system

ẋ1 (t) = x2 (t), x1 (0) = 2

ẋ2 (t) = −2x1 (t) + x2 (t) + u(t), x2 (0) = −3

and the performance index

Z 5
1 2 1
J= [x1 (5) + x1 (5)x2 (5) + 2x2 2 (5)] + [2x21 (t) + 6x1 (t)x2 (t) + 5x22 + 0.25u2 (t)] dt.
2 2 0

4
18. Given a first-order system

ẋ(t) = −2x(t) + u(t)

and the performance index

Z tf
1 2 1
J= x (tf ) + [x2 (t) + u2 (t)] dt.
2 2 0

Find the optimal control.

19. Find the closed-loop optimal control for the first-order system

ẋ(t) = −2x(t) + u(t)

with the performance index

Z ∞
J= [x2 (t) + u2 (t)] dt.
0

20. In the given figure cost of each path is shown. What will be the minimum cost to reach point I, starting from
point A and point C. Form a table for both cases and find the optimal path.

Figure 4

21. An interception problem is described by the following equations:

ẏ = v
v̇ = a

with y, v being the relative separation and relative velocity of the evader-pursuer system and a being
   the ac-
1
  c1 0 y
celeration of the pursuer. Consider the cost functional, to be minimized, given by 2 y v +
0 c2 v t=t
f
R tf 2
t
a dt with c1 , c2 , tf specified as constants. Show that the optimal choice of pursuer acceleration is of the
form a(t) = K1 (t)y(t) + K2 (t)v(t). Obtain the expressions for the time dependent feedback gains in terms of
the constants specified. Hence, clearly explain what happens to these gains under the following special cases:
(i) c1 → 0, (ii) c2 → 0, (iii) c1 → ∞ and c2 → 0. In which of these situations do you have ‘zero miss-distance’ ?
R∞
22. For the first order system ẋ(t) = −x(t) + u(t); x(0) = 1, with a performance measure given by 21 0 e2αt [x2 (t) +
u2 (t)]dt (α > 0), obtain the optimal control and show that the optimal closed-loop system has its pole to the
left of −α.