Scribd
Upload
84 views10 pages

Problems On The Hamiltonian-Jacobi-Bellman Equation: Dr. S. N. Sharma

1) The optimal control is derived from the Hamilton-Jacobi-Bellman equation as the control that minimizes the performance measure. 2) For the system and performance measure given, the optimal control is expressed as a function of the state, time, and partial derivatives of the optimal value function. 3) There are three cases for the optimal control depending on the value of the partial derivative, with the control bounded between -1 and 1.

Uploaded by

Tushar Sanwarey
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
84 views10 pages

Problems On The Hamiltonian-Jacobi-Bellman Equation: Dr. S. N. Sharma

1) The optimal control is derived from the Hamilton-Jacobi-Bellman equation as the control that minimizes the performance measure. 2) For the system and performance measure given, the optimal control is expressed as a function of the state, time, and partial derivatives of the optimal value function. 3) There are three cases for the optimal control depending on the value of the partial derivative, with the control bounded between -1 and 1.

Uploaded by

Tushar Sanwarey
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 10

Subject : Nonlinear Optimal Control

Problems on the Hamiltonian-Jacobi-


Bellman Equation

Dr. S. N. Sharma
Electrical Engineering Department
Sardar Vallabhbhai National Institute Technology, Surat.
Problems on the Hamiltonian-Jacobi-
Bellman Equation
 Example 3.11.1 A first-order system is described by the differential
equation x t  xt  ut ,
where x t and u t are the state and control signal respectively. It
is described to find the control that minimizes the performance
T
1 1 2
measure J  x 2 (T )   u ( t ) dt .
4 0
4

Solution:
 Here the problem is the optimal control that does not account for the
constraints on the state, control and performance measure.
 The constraints on the state, control and the performance measure are
achieved using the bounds or inequalities.
Problems on the Hamiltonian-Jacobi-Bellman
Equation
 A little attention shows that the given problem has
1 2 1
x t  x t  u t , h(t f , x t f )  x (t f ), g (t , x t , u t )  u 2 (t ).
4 4
 Consider the structure of the performance measure in the evolution
form is tf
J (t , xt , u( ) t 0    t f )  h(t f , xt f )   g ( , x , u )d .
t

 As after embedding the optimal control signal, we have the optimal


performance measure tf
J (t , xt )  h(t f , xt f )   g ( , x , u * ( ))d .
*
t

 Now the Hamilton-Jacobi-Bellman equation for the specific form is a


consequence of the general setting.
 *
0  J t (t , x t )  min ( g (t , x t , u (t ))  J x (t , x t ) a (t , x t , u t )) (1)
u (t )
Problems on the Hamiltonian-Jacobi-Bellman
Equation
and  *  * 
(g(t, xt , u(t))  J x (t, xt )a(t, xt , ut ))  g(t, xt , u(t))  J x (t, xt ) a(t, xt , ut )
ut ut ut
 After embedding
1 2 1
x t  x t  u t , h (t f , x t f )  x (t f ), g (t , x t , u t )  u 2 (t ).
4 4
 in the above equation, we get
u (t )
 J x* (t , xt )  0.
2
 Alternatively
u * (t )  2 J x* (t , x t ).
(2)
 After combining equation (2) and the input argument of the min function
of equation (1) and considering the specific system of the given
performance, we arrive at
Problems on the Hamiltonian-Jacobi-Bellman
Equation
And
1 2 * 1 *2 * * *2
min( u (t)  J x (t, xt )(xt  ut ))  u (t)  J x (t, xt )(xt  ut )  J x (t, xt )  J x* (t, xt )xt
u(t ) 4 4

(3)
 From equations (1) and (3), the Hamilton-Jacobi Bellman equation for the
specific case boils down to
 *2
0  Jt (t, xt )  J x (t, xt )  J x* (t, xt )xt .
 And the final value of the optimal performance measure is
tf

J * (t f , xt f )  h(t f , xt f )   g ( , x , u* )d .


tf
Problems on the Hamiltonian-Jacobi-Bellman
Equation
3.7: The first step in using the Hamilton-Jacobi-Bellman equation is to
*
in terms of x(t ), t , J x (t , xt ) that
*
determine the admissible control u (t )
minimizes the performance measure. Find u * (t ) expressed as a
*
function of x (t ), t , J x (t , xt ) for the system

x 1 (t )  x2 (t ), x 2 (t )   x1 (t )  x2 (t )  u (t ),
and the performance measure
T
1
J   (q1 x12 (t )  q 2 x 22 (t )  u 2 (t ))dt, q1 , q 2  0.
0
2
The admissible controls are constrained by u (t )  1.
Solution : Consider the given system
x 1 (t )  x2 (t ), x 2 (t )   x1 (t )  x2 (t )  u (t ), (1a)
Problems on the Hamiltonian-Jacobi-Bellman
Equation
 and the associated performance measure is
T
1
0 2
2 2 2
J  ( q 1 x 1 ( t )  q 2 x 2 ( t )  u (t )) dt ,

 We wish to control the given system that minimizes the performance


measure and obeys the control inequality u (t )  1. For the given
system,
t f  T , g (t , xt , u (t ))  q1 x12 (t )  q 2 x 22 (t )  u 2 (t ),

t f , g (t , xt , u (t )) u (t )
 where the notations and have standard
interpretations of the optimal control.

 The HJB in the non-linear ordinary differential equation setting for the
vector case is
Problems on the Hamiltonian-Jacobi-Bellman
Equation
 and the associated performance measure is
 *T
0  J t (t , x t )  min( g (t , x t , u (t ))  J x (t , x t )a(t , x t , u t )).
u (t )

 Thus, the term within the min function of the right-hand side of the
equation for the given system is
*T 1
g(t, xt , u(t)) J x (t, xt )a(t, xt ,ut ))  (q1x12 (t)  q2 x22 (t)  u2 (t)) J x*1 (t, xt )x2  J x*2 (x1  x2  u(t))
2
 After performing the partial derivative with respect to the control signal
and then setting to zero, we are led to the optimal control signal. Thus,
u  (t )  J x*2  0.
(3)
*
 This implies u  ( t )   J x . 2 After invoking the inequalityu ( t )  1,
1 *
 we have  J x2 (t , x t )  1. The associated optimal performance measure
2
can be rephrased as
Problems on the Hamiltonian-Jacobi-Bellman
Equation
J x*2 (t , x t )  1.
 Alternatively, we construct three cases
 Case 1: u  (t )   J * ,  1  u  (t )  1,  1  J *  1.
x2 x2
 Case 2: Suppose J *  1, then the optimal control signal u* (t) that
1
x2

does not obey the control inequality. Thus, implies u * (t )  1


J x*2  1,
 Case 3: J *  1 impliesu * (t )  1.
x 2

 The Hamilton-Jacobi-Bellman equation becomes

 1
0  J t (t, xt )  (q1 x12 (t)  q2 x22 (t)  J x*1 (t, xt ) x2  J x*2 (x1  x2 )).
2 (2)

 The above equation is an immediate consequence of equations (1a), (2)


and (3).
Problems on the Hamiltonian-Jacobi-Bellman
Equation
 Here we wish to calculate an explicit expression of the given system. A
connection between the general setting abd specific setting, we have
tf T
1 2
J (t , xt )  h(t f , xt f )   g ( , x , u )d   (q1 x12 ( )  q 2 x 22 ( )  u* )d ,
* *

t t
2

 Thus,
1 2
J t* (t , xt )   g (t , xt , u t* )   (q1 x12 (t )  q 2 x22 (t )  J x*2 (t , xt )
2

You might also like