Lect 15 Dynamic Programming
Lect 15 Dynamic Programming
Introduction
Dynamic programming is the mathematical technique dealing with
the optimization of the multistage decision problem
It can be used when the solution to a problem may be viewed as the
result of a sequence of decisions
Complex problems are sometimes solved quickly if approached in a
sequential manner
Dynamic Programming : Sequential or multistage decision making
process
Originated by: Richard Bellman and G.B.Dantzig in 1952
Initially it was known as “ Stochastic Liner Programming”
Basic approach behind dynamic programming: Solution is found out
in multi stages
Let us discuss the Sequential Optimization in Multistage Decision
Process
Let’s represent a Multistage Decision Process
Sequential Optimization
X1 Xt XT
For the tth stage the stage transformation and the benefit
functions are
St+1 = g(Xt , St) and NBt = h(Xt , St)
Representation of Multistage Decision Process
Objective of this multistage problem is to find the optimum values of
all decision variables X1, X2,…, XT such that the individual net benefits
of each stage that is expressed by some objective function, f(NBt )
and the total net benefit which is expressed by
f(NB1, NB2,…, NBT) should be maximized
Introduction
Recursive equations are used to solve a problem in
sequence
These equations are fundamental to the dynamic
programming
Objectives
To formulate recursive equations for a multistage decision
process
In a backward manner and
In a forward manner
Recursive Equations
Recursive equations are used to structure a multistage decision
problem as a sequential process
Each recursive equation represents a stage at which a decision
is required
A series of equations are successively solved, each equation
depending on the output values of the previous equations
A multistage problem is solved by breaking into a number of
single stage problems through recursion
Approached can be done in a backward manner or in a forward
manner
Backward Recursion
A problem is solved by writing equations first for the final
stage and then proceeding backwards to the first stage
Consider a serial multistage problem
NB1 NBt NBT
S1 Stage 1
S2 …. St Stage t
St+1 … ST Stage T
ST+1
X1 Xt XT
Let the
T objective
T function for this problem is
f NBt ht (X t , S t )
t 1 t 1
h1 ( X 1 , S1 ) h2 ( X 2 , S 2 ) ... ht ( X t , S t ) .. hT ( X T , S T ) ....(1)
Backward Recursion …contd.
The relation between the stage variables and decision
variables are St+1 = g(Xt , St) for t = 1,2,…T
Consider the final stage as the first sub-problem. The
input variable to this stage is ST
Principle of optimality: XT should be selected such that is
hT(XT , ST) optimum for the input ST
The objective function for this stage is
f T S T opt hT ( X T , S T )
XT
Backward Recursion …contd.
Next, group the last two stages together as the second
sub-problem.
The objective function is
f T1 S T 1 opt h T 1 ( X T 1 , S T 1 ) hT ( X T , S T )
X T , X T 1
f T1 S T 1 opt hT 1 ( X T 1 , S T 1 ) f T ( g T 1 ( X T 1 , S T 1 ))
X T 1
Backward Recursion …contd.
Thus, a multivariate problem is divided into two single
variable problems as shown
In general, the (i+1)th sub-problem can be expressed as
hT i ( X T i , S T i ) ...
f S T i opt
T i
X T i ...., X T 1 , X T
hT 1 ( X T 1 , S T 1 ) hT ( X T , S T )
Converting this to a single variable problem
f Ti S T i opt hT i ( X T i , S T i ) f T( i 1) ( g T i ( X T i , S T i ))
X T i
Backward Recursion …contd.
f
T i S T i
denotes the optimal value of the objective
X1 Xt XT
Let the objective function for this problem is
T T
f NBt ht (X t , S t )
t 1 t 1
h1 ( X 1 , S1 ) h2 ( X 2 , S 2 ) ... ht ( X t , S t ) .. hT ( X T , S T ) ....(1)
Forward Recursion
The relation between the stage variables and decision
variables are St = g’(Xt+1 , St+1) for t = 1,2,…T
where St is the input available to the stages 1 to t
Consider the stage 1 as the first sub-problem. The input
variable to this stage is S1
Principle of optimality: X1 should be selected such that
h1(X1 , S1) is optimum for the input S1
The objective function for this stage is
f 1 S1 opt h1 ( X 1 , S1 )
X1
Forward Recursion …contd.
Group the first and second stages together as the second
sub-problem.
The objective function is
f 2 S 2 opt h2 ( X 2 , S 2 ) h1 ( X 1 , S1 )
X 2 , X1
hi ( X i , S i ) ...
f i S i opt
X 1 , X 2 ,... X i
h 2 ( X 2 , S 2 ) h1 ( X 1 , S )
1
f i S i opt hi ( X i , S i ) f (i 1) ( g i' ( X i , S i ))
Xi
fi denotes the optimal value of the objective function for
the last i stages.
THANK YOU