Scribd
Upload
125 views25 pages

Dynamic Prog

Dynamic programming is a method for efficiently solving sequential decision problems that exhibit overlapping sub-problems and optimal substructure. It works by breaking down a problem into smaller subproblems and storing the results to avoid recomputing the solutions repeatedly. Dynamic programming can be applied to both discrete and continuous time problems, such as reservoir operation problems, where release decisions must be made sequentially over time. It involves working backwards using recursion to find the optimal policy by maximizing the total benefits over all time periods, while satisfying the system constraints.

Uploaded by

Saranya Uvangelin
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
125 views25 pages

Dynamic Prog

Dynamic programming is a method for efficiently solving sequential decision problems that exhibit overlapping sub-problems and optimal substructure. It works by breaking down a problem into smaller subproblems and storing the results to avoid recomputing the solutions repeatedly. Dynamic programming can be applied to both discrete and continuous time problems, such as reservoir operation problems, where release decisions must be made sequentially over time. It involves working backwards using recursion to find the optimal policy by maximizing the total benefits over all time periods, while satisfying the system constraints.

Uploaded by

Saranya Uvangelin
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 25

DYNAMIC PROGROMMING

Dynamic programming is a method for efficiently solving a broad range of search and optimization problems which exhibit the characteristics of overlappling sub-problems and optimal sub-structure.

Introduction to Dynamic programming


Dynamic programming is ideally suited for sequential decision problems. Sequential decision problems are those in which decisions are made sequentially, one another, based on the state of the system. An example of a sequential decision problem in water resources is reservoir operation problem, in which Release decisions need to be made sequentially

Single Stage Decision Problem

Return R=r(S,x)

Input S

State Transformation T=t(S,X)

Output T

Decision X

Multi Stage-Decision problem


Progress of computations Total Source Q Sluice 1 Sluice 2 Sluice 3

Water allocated R1

Water allocated R2

Water allocated R3

SOLUTION PROCEDURE

Based on Bellmans principle of optimality An optimal policy (a set of decisions) has the property that whatever the initial state and initial decisions are, the remaining decision must constitute an optimal policy with respect to the state resulting from the initial decision. FORWARD Recursion BACKWARD Recursion The computation proceed backwards

Dynamic programming has strong similarities with optimal control, a competing approach to dynamic optimization.

Both dynamic programming and optimal control can be applied to discrete time and continuous time problems, most current applications in Reservoir operation appear to favour dynamic programming for discrete time problems and optimal control for continuous time problems.

Reservoir Operation Steady State policy


Consider a single reservoir receiving inflow it and making releases rt for each time period t Maximum capacity of the reservoir is K Reservoir receives benefits from irrigation, hydropower generation, recreation etc

Optimization problem: Find the sequence of releases to be made from the reservoir that maximizes the total net benefits
St : initial storage for time period t St+1 : final storage for time period t Net benefits can be expressed as a function of rt, St and St+1 as NetBenefit NBt (St , St+1 , rt )

Let there be T periods in a year, then the objective function is to maximize the total net benefits from all periods

The constraints are Continuity Equation after neglecting all the minor losses and assuming no overflow

Backward Recursion in DYNAMIC PROGRAMMING

Take stages as the time periods and the states as the storage volumes.

Assume that there are T periods in a year to find the steady state policy, select a period in a particular year, after which it is assumed that the reservoir is no longer useful (maximum lifetime of the reservoir)
Usually in almost all problems, the last period T is taken as the terminal period.

At this stage, the optimal release will be independent of the inflow and also the net benefit.

Solving this problem in a backward recursion method, let t represents the period in a year from T to 1 and n represents the periods remaining from t till end.
t will take values starting from T, decreasing to 1 (which will complete one year) and then again taking a value of T and repeating the values Value of n starts from 1 (while considering the Tth period of last year) and while moving backwards its value keeps on increasing. i.e. at the beginning of the last year, the value of n = T and at the beginning of second last year its value will be equal to 2T and so on.

Starting from the last period T of a particular year, there is only one period remaining

The maximum net benefit for this last period is

This is solved for all ST values from 0 to K

Considering the last two stages together, the objective function is

In general, for a period t with n periods remaining, the function can be written as

where the index t decreases from T to 1 and then takes the value T again and the cycle repeats for another year the index n starts from 1 and increases at each successive stage

The cycle is repeated till the optimum values of rt for an initial storage St will be the same as the corresponding rt and St of previous year Such a solution is called stationary solution

Maximum net benefit is the difference of

f tn T (S t ) and

for any St and t.

Steady State Policy : Numerical Example

Consider a reservoir for which the desirable constant storage is 20 units and the constant release is 25 units. Capacity of the reservoir is 30 units. Inflows for three seasons are 10, 50 and 20 units.

To find the optimum rt and St that minimizes the total squared deviation from the release and storage targets given.
Thus, the objective function is

St take the discrete values of 0, 10, 20, 30 and rt take the values of 10, 20, 30, 40 Solution: Backward recursion is used to solve the problem Consider a year after which the reservoir is no longer useful. Expressing the problem as a sequential process as shown in the figure below

Considering the last period for which t = 3 and n = 1, the optimization function is
Minimize

Inflow for 3rd season, I3 = 20 units and capacity of the reservoir K = 30 units The release constraints can be expressed as

Computation for the first subproblem (n = 1)

Inflows for three seasons are 10, 50 and 20 units.

Considering the last two periods (n =2), the optimization function is

Inflow for 2nd season, I2 = 50 units Release constraints can be expressed as

While computing the optimal releases for S2=30,

Since r2 can take values only of 10, 20, 30 and 40 only, the release cannot be made for S2=30.

Computation for the second subproblem (n = 2)

The same procedure is repeated for all stages till n = 7. The summarized solution for this problem is given in the table below Solution for the last year

Limitations

Assumption in dynamic programming:

Decisions made at one stage is dependent only on the state variable and is independent of the decisions taken in other stages

You might also like