wrm_ch04.

qxd 8/31/2005 11:51 AM Page 80

4. Optimization Methods
1. Introduction 81

2. Comparing Time Streams of Economic Benefits and Costs 81

2.1. Interest Rates 82
2.2. Equivalent Present Value 82
2.3. Equivalent Annual Value 82

3. Non-Linear Optimization Models and Solution Procedures 83

3.1. Solution Using Calculus 84
3.2. Solution Using Hill Climbing 84
3.3. Solution Using Lagrange Multipliers 86
3.3.1. Approach 86
3.3.2. Meaning of the Lagrange Multiplier 88

4. Dynamic Programming 90
4.1. Dynamic Programming Networks and Recursive Equations 90
4.2. Backward-Moving Solution Procedure 92
4.3. Forward-Moving Solution Procedure 95
4.4. Numerical Solutions 96
4.5. Dimensionality 97
4.6. Principle of Optimality 97
4.7. Additional Applications 97
4.7.1. Capacity Expansion 98
4.7.2. Reservoir Operation 102
4.8. General Comments on Dynamic Programming 112

5. Linear Programming 113

5.1. Reservoir Storage Capacity–Yield Models 114
5.2. A Water Quality Management Problem 117
5.2.1. Model Calibration 118
5.2.2. Management Model 119
5.3. A Groundwater Supply Example 124
5.3.1. A Simplified Model 125
5.3.2. A More Detailed Model 126
5.3.3. An Extended Model 127
5.3.4. Piecewise Linearization Methods 128
5.4. A Review of Linearization Methods 129

6. A Brief Review 132

7. Reference 132

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 81

81

4 Optimization Methods

Optimization methods are designed to provide the ‘best’ values of system design
and operating policy variables – values that will lead to the highest levels of
system performance. These methods, combined with more detailed and accurate
simulation methods, are the primary ways we have, short of actually building
physical models, of estimating the likely impacts of particular water resources
system designs and operating policies. This chapter introduces and illustrates
some of the art of optimization model development and use in analysing water
resources systems. The modelling methods introduced in this chapter are
extended in subsequent chapters.

1. Introduction could be evaluated, on the basis of cost and perhaps other

criteria as well. The best would be called the optimal one, at
All of us are optimizers. We all make decisions that least with respect to the objective criteria used.
maximize our welfare in some way or another. Often the The optimization methods introduced in this chapter
welfare we are maximizing may come later in life. By opti- extend those engineering economics methods. Some are
mizing, it reflects our evaluation of future benefits versus discrete, some are continuous. Continuous optimization
current costs or benefits forgone. In economics, the extent methods can identify the ‘best’ tank design, for example,
to which we value future benefits today is reflected by without having to identify numerous discrete, mutually
what is called a discount rate. While economic criteria are exclusive alternatives. Just how that can be done will be
only a part of everything we consider when making deci- discussed later in this chapter.
sions, they are often among those deemed very important. Before proceeding to a more detailed discussion of
Economic evaluation methods involving discount rates optimization, a review of some methods of dealing with
can be used to consider and compare alternatives charac- time streams of economic incomes or costs (engineering
terized by various benefits and costs that occur over economics) may be useful.
time. This chapter begins with a quick and basic review
of what is often called applied engineering economics. Many
of the optimization methods used in practice incorporate
2. Comparing Time Streams of
concepts from engineering economics. Economic Benefits and Costs
Engineering economic methods typically identify a set of
Alternative plans, p, may involve different benefits and
mutually exclusive alternatives (only one alternative can be
costs over time. These different plans need to be com-
selected) and then, using various methods involving the dis-
pared somehow. Let the net benefit generated in time
count rate, identify the best one. The values of the decision-
period t by plan p be designated simply as B pt . Each plan
variables (e.g., the design and operating policy variables) are
is characterized by the time stream of net benefits it
known for each alternative. For example, consider again the
generates over its planning period Tp.
tank design problem presented in the previous chapter.
Alternative tank designs could be identified, and then each
B 1p, B 2p, B 3p, … , BT pp (4.1)

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 82

82 Water Resources Systems Planning and Management

Clearly, if in any time period t the benefits exceed the costs, the present time. This present worth will depend on
then B tp
0; and if the costs exceed the benefits, B tp  0. the prevailing interest rate in each future time period.
This section defines two ways of comparing different benefit, Assuming an amount V0 is invested at the beginning of
cost or net-benefit time streams produced by different plans a time period, e.g., a year, in a project or a savings account
perhaps having planning periods ending in different years Tp. earning interest at a rate r per period, then at the end of
the period the value of that investment is (1  r)V0.
If one invests an amount V0 at the beginning of period
2.1. Interest Rates
t  1 and at the end of that period immediately reinvests
Fundamental to the conversion of a time series of incomes the total amount (the original investment plus interest
and costs to a single value is the concept of the time value earned), and continues to do this for n consecutive
of money. From time to time, individuals, private corpo- periods at the same period interest rate r, the value, Vn, of
rations and governments need to borrow money to do that investment at the end of n periods would be:
what they want to do. The amount paid back to the lender
Vn  V0(1  r)n (4.2)
has two components: (1) the amount borrowed and (2) an
additional amount called interest. The interest amount is The initial amount V0 is said to be equivalent to Vn at the
the cost of borrowing money, of having the money when end of n periods. Thus the present worth or present value,
it is loaned compared to when it is paid back. In the V0, of an amount of money Vn at the end of period n is:
private sector the interest rate, the added fraction of the V0  Vn/(1  r)n (4.3)
amount loaned that equals the interest, is often identified
as the marginal rate of return on capital. Those who have Equation 4.3 is the basic compound interest discounting
money, called capital, can either use it themselves or they relation needed to determine the present value at the
can lend it to others, including banks, and receive inter- beginning of period 1 of net benefits Vn that accrue at the
est. Assuming people with capital invest their money end of n time periods.
where it yields the largest amount of interest, consistent The total present value of the net benefits generated by
with the risk they are willing to take, most investors plan p, denoted V 0p , is the sum of the values of the net
should be receiving at least the prevailing interest rate as benefits V pt accrued at the end of each time period t times
the return on their capital. the discount factor for that period t. Assuming the inter-
The interest rate includes a number of considerations. est or discount rate r in the discount factor applies for the
One is the time value of money (a willingness to pay duration of the planning period Tp,
Tp
something to obtain money now rather than to obtain the
V 0p  ∑ V tp/ (1 r ) t (4.4)
same amount later). Another is the risk of losing capital
t
(not getting the full amount of a loan or investment
The present value of the net benefits achieved by two or
returned at some future time). A third is the risk of reduced
more plans having the same economic planning horizons
purchasing capability (the expected inflation over time).
Tp can be used as an economic basis for plan selection. If
The greater the risks of losing capital or purchasing
the economic lives or planning horizons of projects differ,
power, the higher the interest rate will be compared to the
then the present value of the plans may not be an appro-
rate reflecting only the time value of money in a secure
priate measure for comparison and plan selection. A valid
and inflation-free environment.
comparison of alternative plans is possible if all plans
cover the same planning period.
2.2. Equivalent Present Value
2.3. Equivalent Annual Value
To compare projects or plans involving different time
series of benefits and costs, it is often convenient to If the lives of various plans differ, but the same plans will
express these time series as a single equivalent value. One be repeated on into the future, then one need only compare
way to do this is to convert the time series to what it is the equivalent constant annual net benefits of each plan.
worth today, its present worth, that is, a single value at Finding the average or equivalent annual amount V p is done

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 83

Optimization Methods 83

in two steps. First, one can compute the present value, V 0p, benefits to those living in future time periods. It can be
of the time stream of net benefits, using Equation 4.4. The viewed as the government’s estimate of the time value of
equivalent constant annual benefits, V p, all discounted to public monies or the marginal rate of return to be
the present must equal the present value, V 0p. achieved by public investments.
Tp Tp
These definitions and concepts of engineering eco-
V 0p  ∑ V tp/(1  r ) t  ∑ V p/ (1  r ) t (4.5) nomics are applicable to many of the problems faced in
t t water resources planning and management. More detailed
discussions of the application of engineering economics
With a little algebra the average annual end-of-year
are contained in numerous texts on the subject.
benefits Vp of the project or plan p is:

Vp  V 0p [r(1  r)Tp]/[(1  r)Tp  1] (4.6)

3. Non-Linear Optimization Models
The capital recovery factor CRFn is the expression and Solution Procedures
[r(1r)Tp]/[(1r)Tp  1] in Equation 4.6 that converts a
fixed payment or present value V 0p at the beginning of n Constrained optimization is also called mathematical
time periods into an equivalent fixed periodic payment V p programming. Mathematical programming techniques include
at the end of each period. If the interest rate per period Lagrange multipliers, linear and non-linear programming,
is r and there are n periods involved, then the capital dynamic programming, quadratic programming, fractional
recovery factor is: programming and geometric programming, to mention a few.
CRFn  [r(1  r)n]/[(1  r)n  1] (4.7) The applicability of each of these as well as other constrained
optimization procedures is highly dependent on the
This factor is often used to compute the equivalent annual mathematical structure of the model. The remainder of this
end-of-year cost of engineering structures that have a chapter introduces and illustrates the application of some
fixed initial construction cost C0 and annual end-of-year of the most commonly used constrained optimization
operation, maintenance, and repair (OMR) costs. The techniques in water resources planning and management.
equivalent uniform end-of-year total annual cost, TAC, These include classical constrained optimization using
equals the initial cost times the capital recovery factor plus calculus-based Lagrange multipliers, discrete dynamic
the equivalent annual end-of-year uniform OMR costs. programming, and linear and non-linear programming.
Consider a river from which diversions are made to
TAC  CRFnC0  OMR (4.8)
three water-consuming firms that belong to the same
For private investments requiring borrowed capital, interest corporation, as illustrated in Figure 4.1. Each firm makes
rates are usually established, and hence fixed, at the time of a product. Water is needed in the process of making that
borrowing. However, benefits may be affected by changing product, and is the critical resource. The three firms can
interest rates, which are not easily predicted. It is common be denoted by the index j  1, 2 and 3 and their water
practice in benefit–cost analyses to assume constant interest allocations by xj. Assume the problem is to determine the
rates over time, for lack of any better assumption. allocations xj of water to each of three firms ( j  1, 2, 3)
Interest rates available to private investors or borrow- that maximize the total net benefits, ∑j NBj(xj), obtained
ers may not be the same rates that are used for analysing from all three firms. The total amount of water available
public investment decisions. In an economic evaluation is constrained or limited to a quantity of Q.
of public-sector investments, the same relationships are Assume the net benefits, NBj(xj), derived from water xj
used even though government agencies are not generally allocated to each firm j, are defined by:
free to loan or borrow funds on private money markets.
In the case of public-sector investments, the interest rate NB1(x1)  6x1  x 12 (4.9)
to be used in an economic analysis is a matter of public NB2(x2)  7x2  1.5x 22 (4.10)
policy; it is the rate at which the government is willing to
forego current benefits to its citizens in order to provide NB3(x3)  8x3  0.5x 32 (4.11)

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 84

84 Water Resources Systems Planning and Management

firm 3
producing ρ 3
firm 1
producing ρ 1
Figure 4.1. Three water-using x3
firms obtain water from river x1
diversions. The amounts re m
ain
Q ing
allocated, xj , to each firm j will riv
er
x2 flow
depend on the amount of water
available, Q, in the river.
firm 2
producing ρ 2

7q
21
11
E0

NB1 ( X1) NB 2 ( X2 ) NB3 ( X3 )

dNB3 / dx 3
dNB1 / dx1
dNB2 / dx 2

Figure 4.2. Concave net benefit

X*
1 X*
2 X*3
functions and their slopes at

r
217
allocations x *, X1 X2 X3
1 x*
2 and x *.
3 11
E0

These are concave functions exhibiting decreasing 3.2. Solution Using Hill Climbing
marginal net benefits with increasing allocations. These
One approach to finding the particular allocations that
functions look like hills, as illustrated in Figure 4.2.
maximize the total net benefit derived from all firms in this
example is an incremental steepest-hill-climbing method.
3.1. Solution Using Calculus
This method divides the total available flow Q into incre-
Calculus can be used to find the allocation to each firm ments and allocates each additional increment so as to get
that maximizes its own net benefit, simply by finding the maximum additional net benefit from that incremental
where the slope or derivative of the net benefit function amount of water. This procedure works in this example
for the firm equals zero. The derivative, dNB(x1)/dx1, of because the functions are concave; in other words, the
the net benefit function for Firm 1 is (6  2x1) and hence marginal benefits decrease as the allocation increases. This
the best allocation to Firm 1 would be 6/2 or 3. The best procedure is illustrated by the flow diagram in Figure 4.3.
allocations for Firms 2 and 3 are 2.33 and 8 respectively. Table 4.1 lists the results of applying the procedure
The total amount of water desired by all firms is the sum shown in Figure 4.3 to the problem of a) allocating 8 and
of each firm’s desired allocation, or 13.33 flow units. b) allocating 20 flow units to the three firms and the river.
However, suppose only 6 units of flow are available for Here a river flow of at least 2 is required and is to be
all three firms. Introducing this constraint renders the satisfied, when possible, before any allocations are made to
previous solution infeasible. In this case we want to the firms.
find the allocations that maximize the total net The hill-climbing method illustrated in Figure 4.3 and
benefit obtained from all firms subject to having only Table 4.1 assigns each incremental flow ∆Q to the use
6 flow units to allocate. Using simple calculus will not that yields the largest additional (marginal) net benefit.
suffice. An allocation is optimal for any total flow Q when the

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 85

Optimization Methods 85

start set initial Q > Q max stop

yes
conditions
no
x 1, x 2 , x 3 , = 0
set Q max, ∆ Q Figure 4.3. Steepest hill-
allocate ∆ Q
to firm j having climbing approach for finding
Q=0
max. marginal Store allocation of a flow Qmax to the
net benefits Q and X three firms, while meeting
dNB j /d x j at x j : data
minimum river flow
Q = Q + ∆Q xj = xj + ∆ Q
requirements R.

Q<R
yes no

7s
21
11
E0

Table 4.1. Hill-climbing iterations for

E020820g
finding allocations that maximize total
Q max = 8; Q i = 0; ∆Q = 1; river flow R > min { Q, 2}
net benefit given a flow of Qmax and a
iteration i Q i allocations. R, x j marginal net new allocations total net required (minimum) streamflow of
benefits benefits R = 2.
Q i + ∆Q
7-3 x2 Σ j NB a( x1 ) j
R x1 x2 x 3 6-2 x1 8- x3 R x1 x2 x3

1-3 0-2 0-2 0 0 0 6 7 8 3 2 0 0 1 7.5

4 3 2 0 0 1 6 7 7 4 2 0 0 2 14.0
5 4 2 0 0 2 6 7 6 5 2 0 1 2 19.5
6 5 2 0 1 2 6 4 6 6 2 0 1 3 25.0
7 6 2 0 1 3 6 4 5 7 2 1 1 3 30.0
8 7 2 1 1 3 4 4 5 8 2 1 1 4 34.5
9 8 2 1 1 4 4 4 4 - - - - - ---

Q max = 20; ∆Q 0; river flow R > min { Q, 2} selected values of Q

Q allocations. R, x j marginal net benefits total net

benefits

R x1 x2 x3 6-2 x1 7-3 x2 8- x3 Σ j NB j ( x1 )

0-2 0-2 0 0 0 6 7 8 0.0

4 2 0 0.25 1.75 6.00 6.25 6.25 14.1
5 2 0.18 0.46 2.36 5.64 5.64 5.64 20.0
8 2 1.00 1.00 4.00 4.00 4.00 4.00 34.5
10 2 1.55 1.36 5.09 2.91 2.91 2.91 41.4
15 2 2.91 2.27 7.82 0.18 0.18 0.18 49.1
20 6.67 3.00 2.33 8.00 0 0 0 49.2

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 86

86 Water Resources Systems Planning and Management

allocation
10
firm 3
8
6 river R
Figure 4.4. Water allocation policy
that maximizes total net benefits 4 firm 1
derived from all three water-using 2 firm 2
firms. 0
0 2 4 6 8 10 12 14 16 18 20

E011217t
Q

marginal net benefits from each non-zero allocation are Let the function Pj(xj) represent the maximum amount
equal, or as close to each other as possible given the size of of product, pj, that can be produced by firm j from an allo-
the increment ∆Q. In this example, with a ∆Q of 1 and cation of water xj. These are called production functions.
Qmax of 8, it just happens that the marginal net benefits are They are typically concave: as xj increases the slope,
all equal (to 4). The smaller the ∆Q, the more optimal will dPj(xj)/dxj, of the production function, Pj(xj), decreases.
be the allocations in each iteration, as shown in the lower For this example assume the production functions for
portion of Table 4.1 where ∆Q approaches 0. the three water-using firms are:
Based on the allocations derived for various values of
p1  0.4(x1)0.9 (4.12)
available water Q, as shown in Table 4.1, an allocation pol-
icy can be defined. For this problem the allocation policy p2  0.5(x2)0.8 (4.13)
that maximizes total net benefits is shown in Figure 4.4.
p3  0.6(x3)0.7 (4.14)
This hill-climbing approach leads to optimal
allocations only if all of the net benefit functions whose Next consider the cost of production. Assume the asso-
sum is being maximized are concave: that is, the marginal ciated cost of production can be expressed by the
net benefits decrease as the allocation increases. following convex functions:
Otherwise, only a local optimum solution can be
C1  3(p1)1.3 (4.15)
guaranteed. This is true using any calculus-based
optimization procedure or algorithm. C2  5(p2)1.2 (4.16)
C3  6(p3)1.15 (4.17)
3.3. Solution Using Lagrange Multipliers Each firm produces a unique patented product, and
3.3.1. Approach hence it can set and control the unit price of its product.
The lower the unit price, the greater the demand and thus
As an alternative to hill-climbing methods, consider the more each firm can sell. Each firm has determined the
a calculus-based method involving Lagrange multipliers. relationship between the amount that can be sold and the
To illustrate this approach, a slightly more complex unit price – that is, the demand functions for that product.
water-allocation example will be used. Assume that the These unit price or demand functions are shown in
benefit, Bj(xj), each water-using firm receives is determined, Figure 4.5 where the pj’s are the amounts of each product
in part, by the quantity of product it produces and the produced. The vertical axis of each graph is the unit price.
price per unit of the product that is charged. As before, To simplify the problem we are assuming linear demand
these products require water and water is the limiting functions, but this assumption is not a necessary condition.
resource. The amount of product produced, pj, by each The optimization problem is to find the water alloca-
firm j is dependent on the amount of water, xj, allocated tions, the production levels and the unit prices that
to it. together maximize the total net benefit obtained from all

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 87

Optimization Methods 87

unit price ($)

32 32 32
28 28 28
24 24 24
20 20 20 Figure 4.5. Unit prices that
16 16 16 28 - 2.5 ρ3 will guarantee the sale of the
20 - 1.5 ρ2
12 12 12 specified amounts of products pj
8 12 - ρ1 8 8 produced in each of the three
4 4 4 firms.
0 0 0
ρ ρ ρ
E011217u

1 2 3

three firms. The water allocations plus the amount that One can first solve this model for the values of each pj that
must remain in the river, R, cannot exceed the total maximize the total net benefits, assuming water is not
amount of water Q available. a limiting constraint. This is equivalent to finding each
Constructing and solving a model of this problem for individual firm’s maximum net benefits, assuming all the
various values of Q, the total amount of water available, will water that is needed is available. Using calculus we can
define the three allocation policies as functions of Q. These equate the derivatives of the total net benefit function
policies can be displayed as a graph, as in Figure 4.4, with respect to each pj to 0 and solve each of the resulting
showing the three best allocations given any value of Q. three independent equations:
This of course assumes the firms can adjust to varying
Total Net_benefit  [(12  p1)p1  (20  1.5p2)p2
values of Q. In reality this may not be the case. (Chapter 10
 (28  2.5p3)p3]  [3(p1)1.30
examines this problem using more realistic benefit
 5(p2)1.20  6(p3)1.15] (4.26)
functions that reflect the degree to which firms can adapt
to changing inputs over time.) Derivatives:

The model: ∂(Net_benefit)/∂p1  0  12  2p1  1.3(3)p 10.3 (4.27)

Maximize Net_benefit (4.18) ∂(Net_benefit)/∂p2  0  20  3p2  1.2(5)p 20.2 (4.28)

Subject to: ∂(Net_benefit)/∂p3  0  28  5p3  1.15(6)p 0.15

3
Definitional constraints: (4.29)
Net_benefit  Total_return  Total_cost (4.19) The result (rounded off ) is p1  3.2, p2  4.0, and
p3  3.9 to be sold for unit prices of 8.77, 13.96, and
Total_return  (12  p1)p1  (20  1.5p2)p2
18.23 respectively, for a maximum net revenue of 155.75.
 (28  2.5p3)p3 (4.20)
This would require water allocations x1  10.2, x2  13.6
Total_cost  3(p1)1.30  5(p2)1.20  6(p3)1.15 (4.21) and x3  14.5, totaling 38.3 flow units. Any amount of
water less than 38.3 will restrict the allocation to, and
Production functions defining the relationship between
hence the production at, one or more of the three firms.
water allocations xj and production pj:
If the total available amount of water is less than that
p1  0.4(x1)0.9 (4.22) desired, constraint Equation 4.25 can be written as an
equality, since all the water available, less any that must
p2  0.5(x2)0.8 (4.23)
remain in the river, R, will be allocated. If the available
p3  0.6(x3)0.7 (4.24) water supplies are less than the desired 38.3 plus the
required streamflow R, then Equations 4.22 through 4.25
Water allocation restriction: need to be added. These can be rewritten as equalities
R  x1  x2  x3  Q (4.25) since they will be binding.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 88

88 Water Resources Systems Planning and Management

p1  0.4(x1)0.9  0 (4.30) constraint, Equation 4.33, the amount of water available,

Q. We will prove this shortly.
p2  0.5(x2)0.8  0 (4.31) Differentiating Equation 4.34 with respect to each of
p3  0.6(x3)0.7  0 (4.32) the ten unknowns and setting the resulting equations to
0 yields:
R  x1  x2  x3  Q  0 (4.33)
1  λ1
∂L/∂p1  0  12  2p1  1.3(3)p 0.3 (4.35)
The first three constraints, Equations 4.30, 4.31 and 4.32,
are the production functions specifying the relationships 2  λ2
∂L/∂p2  0  20  3p2  1.2(5)p 0.2 (4.36)
between each water input xj and product output pj. The ∂L/∂p3  0  28  5p3  1.15(6)p 30.15  λ3 (4.37)
fourth constraint Equation 4.33 is the restriction on the
total allocation of water. Since each of the four constraint ∂L/∂x1  0  λ10.9(0.4)(x1)0.1  λ4 (4.38)
equations equals zero, each can be added to the net ben- ∂L/∂x2  0  λ20.8(0.5)(x2)0.2  λ4 (4.39)
efit Equation 4.26 without changing its value. This is
∂L/∂x3  0  λ30.7(0.6)(x3)0.3  λ4 (4.40)
done in Equation 4.34, which is equivalent to Equation
4.26. The variable L is the value of the Lagrange form of ∂L/∂λ1  0  p1  0.4(x1)0.9 (4.41)
the objective function: ∂L/∂λ2  0  p2  0.5(x2)0.8 (4.42)
L  [(12  p1)p1  (20  1.5p2)p2  (28  2.5p3)p3] ∂L/∂λ3  0  p3  0.6(x3)0.7 (4.43)
 [3(p1)1.30  5(p2)1.20  6(p3)1.15]
∂L/∂λ4  0  R  x1  x2  x3  Q (4.44)
 λ1[p1  0.4(x1)0.9]  λ2[p2  0.5(x2)0.8]
These ten equations are the conditions necessary for an
 λ3[p3  0.6(x3)0.7]  λ4[R  x1  x2  x3  Q]
optimal solution. They can be solved to obtain the values of
(4.34)
the ten unknown variables. The solutions to these equations
Since each of the four constraint Equations 4.30 through for various values of Q, (found in this case by using LINGO)
4.33 included in Equation 4.34 equals zero, each can be are shown in Table 4.2. (A demo version of LINGO is
multiplied by a variable λ i without changing the value of available, together with its help files, at www.lindo.com.)
Equation 4.34. These unknown variables λ i are called the
Lagrange multipliers of constraints i. The value of each
3.3.2. Meaning of the Lagrange Multiplier
multiplier, λi, is the marginal value of the original objec-
tive function, Equation 4.26, with respect to a change in In this example, Equation 4.34 is the objective function.
the value of the amount produced, pi, or in the case of It is maximized (or minimized) by equating to zero each

Table 4.2. Solutions to Equations 4.35 E020820h

through 4.44.
water allocations product Lagrange multipliers
to firms productions
available 1 2 3 1 2 3 marginal net benefits

Q-R x1 x2 x3 p1 p2 p3 λ1 λ2 λ3 λ4

10 1.2 3.7 5.1 0.46 1.44 1.88 8.0 9.2 11.0 2.8
20 4.2 7.3 8.5 1.46 2.45 2.68 4.7 5.5 6.6 1.5
30 7.5 10.7 11.7 2.46 3.34 3.37 2.0 2.3 2.9 0.6
38 10.1 13.5 14.4 3.20 4.00 3.89 0.1 0.1 0.1 0.0
38.3 10.2 13.6 14.5 3.22 4.02 3.91 0 0 0 0

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 89

Optimization Methods 89

of its partial derivatives with respect to each unknown solution, one needs to check whether in fact a maximum
variable. Equation 4.34 consists of the original net benefit or minimum is being obtained. In this example, since
function plus each constraint i multiplied by a weight each net benefit function is concave, a maximum will
or multiplier λ i. This equation is expressed in monetary result.
units, such as dollars or euros. The added constraints are The meaning of the values of the multipliers λ i at
expressed in other units: either the quantity of product the optimum solution can be derived by manipulation
produced or the amount of water available. Thus the units of Equation 4.48. Taking the partial derivative of the
of the weights or multipliers λ i associated with these Lagrange function, Equation 4.47, with respect to an
constraints are expressed in monetary units per constraint unknown variable xj and setting it to zero results in:
units. In this example the multipliers λ1, λ 2 and λ 3
∂L/∂xj  0  ∂F/∂xj  ∑λ i∂(gi(X))/∂xj (4.49)
represent the change in the total net benefit value of i
the objective function (Equation 4.26) per unit change in Multiplying each term by ∂xj yields
the products p1, p2 and p3 produced. The multiplier λ4
represents the change in the total net benefit per unit ∂F  ∑λ i∂(gi(X)) (4.50)
i
change in the water available for allocation, Q  R. Dividing each term by ∂bk associated with a particular
Note in Table 4.2 that as the quantity of available constraint, say k, defines the meaning of λ k.
water increases, the marginal net benefits decrease. This is
reflected in the values of each of the multipliers, λ i. In ∂F/∂bk  ∑λ i∂(gi(X))/∂bk  λ k (4.51)
i
other words, the net revenue derived from a quantity of Equation 4.51 follows from the fact that ∂( gi(X))/∂bk  0
product produced at each of the three firms, and from the for constraints i  k and ∂(gi(X))/∂bk  1 for the
quantity of water available, are concave functions of those constraint i  k. The latter is true since bi  gi(X) and thus
quantities, as illustrated in Figure 4.2. ∂(gi(X))  ∂bi.
To review the general Lagrange multiplier approach Thus from Equation 4.51, each multiplier λ k is the
and derive the definition of the multipliers, consider the marginal change in the original objective function F(X) with
general constrained optimization problem containing n respect to a change in the constant bk associated with the
decision-variables xj and m constraint equations i. constraint k. For non-linear problems it is the slope of the
Maximize (or minimize) F(X) (4.45) objective function plotted against the value of bk.
subject to constraints Readers can work out a similar proof if a slack or
surplus variable, Si, is included in inequality constraints
gi(X)  bi i 1, 2, 3, … , m (4.46)
to make them equations. For a less-than-or-equal
where X is the vector of all xj. The Lagrange function L(X, λ) constraint gi(X)  bi a squared slack variable S 2i can be
is formed by combining Equations 4.46, each equalling added to the left-hand side to make it an equation gi(X) 
zero, with the objective function of Equation 4.45. S i2  bi . For a greater-than-or-equal constraint gi(X)  bi a
squared surplus variable S i2 can be subtracted from the
L(X, λ)  F(X)  ∑λi(gi(X)  bi) (4.47)
i left hand side to make it an equation gi(X)  S 2i  bi.
Solutions of the equations These slack or surplus variables are squared to ensure
∂L/∂xj  0 for all decision-variables j they are non-negative, and also to make them appear in
and the differential equations.
∂L/∂λi  0 for all constraints i (4.48) ∂L /∂Si  0  2Siλi  Siλi (4.52)
are possible local optima. Equation 4.52 shows that either the slack or surplus
There is no guarantee that a global optimum solution will variable, Si, or the multiplier, λ i, will always be zero. If the
be found using calculus-based methods such as this one. value of the slack or surplus variable Si is non-zero, the
Boundary conditions need to be checked. Furthermore, constraint is redundant. The optimal solution will not be
since there is no difference in the Lagrange multipliers affected by the constraint. Small changes in the values, bi,
procedure for finding a minimum or a maximum of redundant constraints will not change the optimal value

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 90

90 Water Resources Systems Planning and Management

of the objective function F(X). Conversely, if the constraint 4.1. Dynamic Programming Networks and
is binding, the value of the slack or surplus variable Si will Recursive Equations
be zero. The multiplier λi can be non-zero if the value of
the function F(X) is sensitive to the constraint value bi. A network of nodes and links can represent each discrete
The solution of the set of partial differential Equations dynamic programming problem. Dynamic programming
4.52 often involves a trial-and-error process, equating to methods find the best way to get to any node in that
zero a λi or a Si for each inequality constraint and solving the network. The nodes represent possible discrete states that
remaining equations, if possible. This tedious procedure, can exist and the links represent the decisions one could
along with the need to check boundary solutions when non- make to get from one state to another. Figure 4.6 illustrates
negativity conditions are imposed, detracts from the utility a portion of such a network for the three-firm allocation
of classical Lagrange multiplier methods for solving all but problem shown in Figure 4.1. In this case the total amount
relatively simple water resources planning problems. of water available, Q  R, to all three firms is 10.
Thus, dynamic programming models involve states,
stages and decisions. The relationships among states,
4. Dynamic Programming stages and decisions are represented by networks, such as
that shown in Figure 4.6. The states of the system are the
The water allocation problems in the previous section nodes and the values of the states are the numbers in the
considered a net-benefit function for each water-using nodes. Each node value in this example is the quantity of
firm. In those examples they were continuous differen- water available to allocate to all remaining firms, that
tiable functions, a convenient attribute if methods based is, to all connected links to the right of the node. These
on calculus (such as hill-climbing or Lagrange multipliers) state variable values typically represent some existing
are to be used to find the best solution. In many practical condition either before making, or after having made, a
situations these functions may not be so continuous, or decision. The stages of the system are the separate
so conveniently concave for maximization or convex for columns of linked nodes. The links in this example
minimization, making calculus-based methods for their represent possible allocation decisions for each of the
solution difficult. three different firms. Each stage is a separate firm.
A possible solution method for constrained
optimization problems containing continuous and/or x1 x2 x3

discontinuous functions of any shape is called discrete 7 4 3

dynamic programming. Each decision-variable value can
3 5
assume one of a set of discrete values. For continuous 0
6
4
valued objective functions, the solution derived from
5
discrete dynamic programming may therefore be only an 6 4 2
0
approximation of the best one. For all practical purposes 3 5
0 9
this is not a significant limitation, especially if the intervals 4
6
2
between the discrete values of the decision-variables are 5
5 4
not too large and if simulation modelling is used to refine
3 5
the solutions identified using dynamic programming. 8
4
Dynamic programming is an approach that divides the
original optimization problem, with all of its variables, 4 4 0

into a set of smaller optimization problems, each of which

7w

firm 1 firm 2 firm 3

21

11
needs to be solved before the overall optimum solution to E0

the original problem can be identified. The water supply

Figure 4.6. A network representing some of the possible
allocation problem, for example, needs to be solved for a
integer allocations of water to three water-consuming firms j.
range of water supplies available to each firm. Once this The circles or nodes represent the discrete quantities of
is done the particular allocations that maximize the total water available, and the links represent feasible allocation
net benefit can be determined. decisions xj.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 91

Optimization Methods 91

Each link connects two nodes, the left node value NB 1 ( x 1) NB 2 ( x 2 ) NB 3 ( x 3 )

indicating the state of a system before a decision is made,
and the right node value indicating the state of a system after 7 27.9 3

a decision is made. In this case the state of the system is the 5.7 3
0
amount of water available to allocate to the remaining firms. 8.6
33.7

In the example shown in Figure 4.6, the state and 2

6 27.9 2
decision-variables are represented by integer values – an 0.0
5.7 3
admittedly fairly coarse discretization. The total amount 0 3.7 9
8.6 33.7
of water available, in addition to the amount that 6.3
2
must remain in the river, is 10. Note from the first row of 5 27.9

Table 4.2 the exact allocation solution is x1  1.2, 5.7 3

x2  3.7, and x3  5.1. Normally we wouldn’t know this 8

8.6
solution before solving for it using dynamic program-
4 27.9 0
ming, but since we do we can reduce the complexity of
the dynamic programming network so that the repetitive

7x
firm 1 firm 2 firm 3

21
11
E0
process of finding the best solution is clearer. Thus
assume the range of x1 is limited to integer values from Figure 4.7. Network representing integer value allocations of
0 to 2, the range of x2 is from 3 to 5, and the range of x3 water to three water-consuming firms. The circles or nodes
is from 4 to 6. These range limits are imposed here just to represent the discrete quantities of water available, and the
reduce the size of the network. In this case, these assump- links represent feasible allocation decisions. The numbers on
tions will not affect or constrain the optimal solution. If the links indicate the net benefits obtained from these
particular allocation decisions.
we did not make these assumptions the network would
have, after the first column of one node, three columns
of 11 nodes, one representing each integer value from 0 quantity Q  R  10. Our task is to find the best path
to 10. Finer (non-integer) discretizations would involve through the network, beginning at the left-most node
even more nodes and connecting links. having a state value of 10. To do this we need to know the
The links of Figure 4.6 represent the water allocations. net benefits we will get associated with all the links (rep-
Note that the link allocations, the numbers on the links, resenting the allocation decisions we could make) at each
cannot exceed the amount of water available, that is, the node (state) for each firm (stage).
number in the left node. The number in the right node is Figure 4.7 shows the same network as in Figure 4.6;
the quantity of water remaining after an allocation has however the numbers on the links represent the net ben-
been made. The value in the right node, state Sj1, at the efits obtained from the associated water allocations. For
beginning of stage j1, is equal to the value in the left the three firms j  1, 2 and 3, the net benefits, NBj(xj),
node, Sj, less the amount of water, xj, allocated to firm j as associated with allocations xj are:
indicated on the link. Hence, beginning with a quantity of NB1(x1)  maximum(12  p1)p1  3(p1)1.30
water Q  R that can be allocated to all three firms, after where p1  0.4(x1)0.9 (4.56)
allocating x1 to Firm 1 what remains is S2:
NB2(x2)  maximum(20  1.5p2)p2  5(p2)1.20
Q  R  x1  S2 (4.53)
where p2  0.5(x2)0.8 (4.57)
Allocating x2 to Firm 2, leaves S3. NB3(x3)  maximum(28  2.5p3)p3  6(p3)1.15
S2  x2  S3 (4.54) where p3  0.6(x3)0.7 (4.58)
Finally, allocating x3 to Firm 3 leaves S4. respectively.
The discrete dynamic programming algorithm or pro-
S3  x3  S4 (4.55)
cedure is a systematic way to find the best path through
Figure 4.6 shows the different values of each of these this network, or any other suitable network. What makes
states, Sj, and decision-variables xj beginning with a a network suitable for dynamic programming is the fact

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 92

92 Water Resources Systems Planning and Management

that all the nodes can be lined up in a sequence of find the overall best set of decisions for each stage: in
columns and each link connects a node in one column to other words, the best allocations to each firm (represented
another node in the next column of nodes. No link passes by the best path through the network) in this example.
over or through any other column(s) of nodes. Links also Dynamic programming networks can be solved in
do not connect nodes in the same column. In addition, two ways – beginning at the most right column of nodes
the contribution to the overall objective value (in this or states and moving from right to left, called the
case, the total net benefits) associated with each discrete backward-moving (but forward-looking) algorithm, or
decision (link) in any stage or for any firm is strictly a beginning at the left most node and moving from left to
function of the allocation of water to the firm. It is right, called the forward-moving (but backward-looking)
not dependent on the allocation decisions associated with algorithm. Both methods will find the best path through
other stages (firms) in the network. the network. In some problems, however, only the
The main challenge in using discrete dynamic backward-moving algorithm produces a useful solution.
programming to solve an optimization problem is to struc- This is especially relevant when the stages are time
ture the problem so that it fits this dynamic programming periods. We often want to know what we should do next
network format. Perhaps surprisingly, many water resources given a particular state we are in, not what we should
planning and management problems do. But it takes have just done to get to the particular state we are in. We
practice to become good at converting optimization cannot alter past decisions, but we can, and indeed must,
problems to networks of states, stages and decisions suitable make future decisions. We will revisit this issue when
for solution by discrete dynamic programming algorithms. we get to reservoir operation where the stages are time
In this problem the overall objective is to: periods.
3
Maximize ∑ NBj(xj) (4.59)
j 1 4.2. Backward-Moving Solution Procedure
where NBj(xj) is the net benefit associated with an Consider the network in Figure 4.7. Again, the nodes
allocation of xj to firm j.. Equations 4.56, 4.57 and 4.58 represent the discrete states – the water available to allocate
define these net benefit functions. As before, the index j to all remaining users. The links represent particular
represents the particular firm, and each firm is a stage for discrete allocation decisions. The numbers on the links are
this problem. Note that the index or subscript used in the the net benefits obtained from those allocations. We want
objective function often represents an object (like a water- to proceed through the node-link network from the state of
using firm) at a place in space or in a time period. These 10 at the beginning of the first stage to the end of the
places or time periods are called the stages of a dynamic network in such a way as to maximize total net benefits.
programming problem. Our task is to find the best path But without looking at all combinations of successive
from one stage to the next: in other words, the best allocations we cannot do this beginning at a state of 10.
allocation decisions for all three firms. However, we can find the best solution if we assume we
Dynamic programming is called a multi-stage have already made the first two allocations and are at any
decision-making process. Instead of deciding all three of the nodes or states at the beginning of the final, third,
allocations in one single optimization procedure, like stage with only one allocation decision remaining. Clearly
Lagrange multipliers, the dynamic programming proce- at each node representing the water available to allocate to
dure divides the problem up into many optimization the third firm, the best decision is to pick the allocation
problems, one for each possible discrete state (e.g., (link) having the largest net benefits.
amount of water available) in each stage (e.g., for each Denoting F3(S3) as the maximum net benefits we can
firm). Given a particular state Sj and stage j – that is, a achieve from the remaining amount of water S3, then for
particular node in the network – what decision (link) xj each discrete value of S3 we can find the x3 that maximizes
will result in the maximum total net benefits, designated F3(S3). Those shown in Figure 4.7 include:
as Fj(Sj), given this state Sj for this and all remaining stages
or firms j, j  1, j  2 … ? This question must be F3(7)  Maximum{NB3(x3)}
answered for each node in the network before one can x3  7, the total flow available.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 93

Optimization Methods 93

4  x3  6, the allowable range of allocation

F 3 (7)
decisions. = 33.7

 Maximum{27.9, 31.1, 33.7}  33.7 F 2 (10) =

7 27.9 3

when x3  6 (4.60) 18.6 + 33.7

= 52.3 5.7 3
0
F 3 (6) 33.7
F3(6)  Maximum{NB3(x3)}
8.6
= 33.7
2
x3  6 F 1 (10) = F 2 (9) = 6 27.9 2
3.7 + 49.7 0.0 18.6 + 31.1
3
4  x3  6 = 53.4 = 49.7 5.7
0 3.7 9
8.6 F 3 (5) 33.7
 Maximum{27.9, 31.1, 33.7}  33.7 6.3 = 31.1
2
when x3  6 (4.61) 5 27.9

5.7 3
F3(5)  Maximum{NB3(x3)} 8
8.6 F 3 (4)
x3  5 F 2 (8) = = 27.9
15.7 + 31.1
4 27.9 0
4  x3  6 = 46.8

7y
 Maximum{27.9, 31.1}  31.1 firm 1 firm 2 firm 3

21
11
E0

when x3  5 (4.62)
Figure 4.8. Using the backward-moving dynamic program-
F3(4)  Maximum{NB3(x3)} ming method for finding the maximum remaining net benefits,
x3  4 Fj(Sj), and optimal allocations (denoted by the arrows on the
links) for each state in Stage 3, then for each state in Stage 2
4  x3  6
and finally for the initial state in Stage 1 to obtain the optimum
 Maximum{27.9}  27.9 when x3  4 (4.63) allocation policy and maximum total net benefits, F1(10). The
minimum flow to remain in the river, R, is in addition to the ten
These computations are shown on the network in units available for allocation and is not shown in this network.
Figure 4.8. Note that there are no benefits to be obtained
after the third allocation, so the decision to be made for x2  10
each node or state prior to allocating water to Firm 3 is
simply that which maximizes the net benefits derived 3  x2  5
from that last (third) allocation. In Figure 4.8 the links  Maximum{15.7  33.7, 18.6  33.7,
representing the decisions or allocations that result in the 21.1  31.1}  52.3 when x2  4
largest net benefits are shown with arrows. F2(9)  Maximum{NB2(x2)  F3(S3  9  x2)} (4.65)
Having computed the maximum net benefits, F3(S3), x2  9
associated with each initial state S3 for Stage 3, we can now
3  x2  5
move backward (to the left) to the discrete states S2 at the
beginning of the second stage. Again, these states represent  Maximum{15.7  33.7, 18.6  31.1,
the quantity of water available to allocate to Firms 2 and 3. 21.1  27.9}  49.7 when x2  4
Denote F2(S2) as the maximum total net benefits obtained F2(8)  Maximum{NB2(x2)  F3(S3  8  x2)} (4.66)
from the two remaining allocations x2 and x3 given the x2  8
quantity S2 water available. The best x2 depends not only
3  x2  5
on the net benefits obtained from the allocation x2 but also
on the maximum net benefits obtainable after that, namely  Maximum{15.7  31.1, 18.6  27.9}  46.8
the just-calculated F3(S3) associated with the state S3 that when x2  3
results from the initial state S2 and a decision x2. As These maximum net benefit functions, F2(S2), could be
defined in Equation 4.54, this final state S3 in Stage 2 obvi- calculated for the remaining discrete states from 7 to 0.
ously equals S2  x2. Hence for those nodes at the begin- Having computed the maximum net benefits obtain-
ning of Stage 2 shown in Figure 4.8: able for each discrete state at the beginning of Stage 2,
that is, all the F2(S2) values, we can move backward or left
F2(10)  Maximum{NB2(x2)  F3(S3  10 x2)} (4.64) to the beginning of Stage 1. For this problem there is only

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 94

94 Water Resources Systems Planning and Management

one state, the state of 10 we are actually in before making The discrete dynamic programming version of this
any allocations to any of the firms. In this case, the maxi- problem required discrete states Sj representing the
mum net benefits, F1(10), we can obtain from all three amount of water available to allocate to firms j, j  1, … , .
allocations given 10 units of water available, is It required discrete allocations xj. Next it required the
calculation of the maximum net benefits, Fj(Sj), that could
F1(10)  Maximum{NB1(x1)  F2(S2  10  x1)} (4.67)
be obtained from all firms j, beginning with Firm 3, and
x1  10 proceeding backwards as indicated in Equations 4.76
0  x1  2 to 4.78.
 Maximum{0  52.3, 3.7  49.7, F3(S3)  maximum{NB3(x3)} over all x3  S3, for all
6.3  46.8}  53.4 when x1  1 discrete S3 values between 0 and 10 (4.76)
Equation 4.67 is another way of expressing Equation F2(S2)  maximum{NB2(x2)  F3(S3)} over all x2  S2
4.59. Their values are the same. It is the maximum net and S3  S2  x2, 0  S2  10 (4.77)
benefits obtainable from allocating the available 10 units
of water. From Equation 4.67 we know that we will get a F1(S1)  maximum{NB1(x1)  F2(S2)} over all x1  S1
maximum of 53.4 net benefits if we allocate 1 unit of and S2  S1  x1 and S1  10 (4.78)
water to Firm 1. This leaves 9 units of water to allocate to To solve for F1(S1) and each optimal allocation xj we must
the two remaining firms. This is our optimal state at the first solve for all values of F3(S3). Once these are known
beginning of Stage 2. Given a state of 9 at the beginning we can solve for all values of F2(S2). Given these F2(S2)
of Stage 2, we see from Equation 4.65 that we should allo- values, we can solve for F1(S1). Equations 4.76 need to
cate 4 units of water to Firm 2. This leaves 5 units of be solved before Equations 4.77 can be solved, and
water for Firm 3. Given a state of 5 at the beginning Equations 4.77 need to be solved before Equations 4.78
of Stage 3, Equation 4.62 tells us we should allocate all can be solved. They need not be solved simultaneously,
5 units to Firm 3. All this is illustrated in Figure 4.8. and they cannot be solved in reverse order. These three
Compare this discrete solution with the continuous equations are called recursive equations. They are defined
one defined by Lagrange multipliers as shown in for the backward-moving dynamic programming solution
Table 4.2. The exact solution, to the nearest tenth, is 1.2, procedure.
3.7, and 5.1 for x1, x2 and x3 respectively. The solution There is a correspondence between the non-linear
just derived from discrete dynamic programming that optimization model defined by Equations 4.68 to 4.75
assumed only integer values is 1, 4, and 5 respectively. and the dynamic programming model defined by the
To summarize, a dynamic programming model was recursive Equations 4.76, 4.77 and 4.78. Note that F3(S3)
developed for the following problem: in Equation 4.76 is the same as:
Maximize Net_benefit (4.68) F3(S3)  Maximum NB3(x3) (4.79)
Subject to: Subject to:
Net_benefit  Total_return  Total_cost (4.69) x3  S3 (4.80)
Total_return  (12  p1)p1  (20  1.5p2)p2 where NB3(x3) is defined in Equation 4.58.
 (28  2.5p3)p3 (4.70)
Similarly, F2(S2) in Equation 4.62 is the same as:
Total_cost  3(p1)1.30  5(p2)1.20  6(p3)1.15 (4.71)
F2(S2)  Maximum NB2(x2)  NB3(x3) (4.81)
p1  0.4(x1)0.9 (4.72)
Subject to:
p2  0.5(x2)0.8 (4.73) x2  x3  S2 (4.82)
p3  0.6(x3) 0.7
(4.74)
where NB2(x2) and NB3(x3) are defined in Equations 4.57
x1  x2  x3  10 (4.75) and 4.58.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 95

Optimization Methods 95

Finally, F1(S1) in Equation 4.63 is the same as: objective function that can be obtained from all further or
F1(S1)  Maximum NB1(x1)  NB2(x2)  NB3(x3) (4.83) remaining decisions. Alternatively one can proceed forward,
that is, from left to right, through a dynamic programming
Subject to: network. For the forward-moving algorithm at each node
x1  x2  x3  S1  10 (4.84) we need to calculate the best value of the objective function
where NB1(x1), NB2(x2) and NB3(x3) are defined in that could be obtained from all past decisions leading to that
Equations 4.56, 4.57 and 4.58. node or state. In other words, we need to find how best to
Alternatively, F3(S3) in Equation 4.76 is the same as: get to each state Sj1 at the end of each stage j.
Returning to the allocation example, define fj(Sj1) as
F3(S3)  Maximum(28  2.5p3)p3  6(p3)1.15 (4.85) the maximum net benefits from the allocation of water to
Subject to: firms 1, 2, … , j, given a state Sj1 after having made those
p3  0.6(x3)0.7 (4.86) allocations. For this example we begin the forward-
moving, but backward-looking, process by selecting each
x3  S3 (4.87)
of the ending states in the first stage j  1 and finding the
Similarly, F2(S2) in Equation 4.77 is the same as: best way to have arrived at (or to have achieved) those
F2(S2)  Maximum(20  1.5p2)p2  (28  2.5p3) ending states. Since in this example there is only one way
p3  5(p2)1.20  6(p3)1.15 (4.88) to get to each of those states, as shown in Figure 4.6 or
4.7, the allocation decisions are obvious.
Subject to:
f1(S2)  maximum{NB1(x1)}
p2  0.5(x2)0.8 (4.89)
x1  10  S2 (4.98)
p3  0.6(x3)0.7 (4.90)
Hence, f1(S2) is simply NB1(10  S2). Once the values
x2  x3  S2 (4.91) for all f1(S2) are known for all discrete S2 between 0 and 10,
Finally, F1(S1) in Equation 4.78 is the same as: move forward (to the right) to the end of Stage 2 and find
the best allocations to have made given each final state S3.
F1(S1)  Maximum(12  p1)p1  (20  1.5p2)p2
 (28  2.5p3)p3 f2(S3)  maximum{NB2(x2)  f1(S2)}
 [3(p1)1.30  5(p2)1.20  6(p3)1.15] (4.92) 0  x2  10  S3
Subject to: S2  S3  x2 (4.99)
p1  0.4(x1) 0.9
(4.93) Once the values of all f2(S3) are known for all discrete states
p2  0.5(x2) 0.8
(4.94) S3 between 0 and 10, move forward to Stage 3 and find the
best allocations to have made given each final state S4.
p3  0.6(x3) 0.7
(4.95)
x1  x2  x3  S1  10 (4.96) f3(S4)  maximum{NB3(x3)  f2(S3)}
for all discrete S4 between 0 and 10.
The transition function of dynamic programming defines
the relationship between two successive states Sj and Sj1 0  x3  10  S4
and the decision xj. In the above example these transition S3  S4  x3 (4.100)
functions are defined by Equations 4.53, 4.54 and 4.55, Figure 4.9 illustrates a portion of the network represented
or, in general terms for all firms j, by: by Equations 4.98 through 4.100, and the fj(Sj1) values.
Sj1  Sj  xj (4.97) From Figure 4.9, note the highest total net benefits are
obtained by ending with 0 remaining water at the end of
Stage 3. The arrow tells us that if we are to get to that state
4.3. Forward-Moving Solution Procedure
optimally, we should allocate 5 units of water to Firm 3.
We have just described the backward-moving dynamic Thus we must begin Stage 3, or end Stage 2, with
programming algorithm. In that approach at each node 10  5  5 units of water. To get to this state at the end
(state) in each stage we calculated the best value of the of Stage 2 we should allocate 4 units of water to Firm 2.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 96

96 Water Resources Systems Planning and Management

programs available that will solve all dynamic program-

F 2 (7)
= 15.7 ming problems. Thus any user of dynamic programming
F 1 (10) 7 27.9 3 will need to write a computer program to solve a particu-
=0
5.7 3 F 3 (3) lar problem. Most computer programs written for solving
0
8.6
F 2 (6) 33.7
= 43.6 specific dynamic programming problems create and store
2
= 19.4
the solutions of the recursive equations in tables. Each
6 27.9 2
0.0
F 1 (9) stage is a separate table, as shown in Tables 4.3, 4.4 and
= 3.7
3
5.7 F 3 (2)
= 47.3
4.5 for this example water allocation problem. These
0 3.7 9
F 2 (5) 33.7
8.6
= 22.3
tables apply to only a part of the entire problem, namely
6.3
2
5 27.9 that part of the network shown in Figures 4.8 and 4.9. The
F 1 (8)
= 6.3
5.7 3 F 3 (1)
backward solution procedure is used.
8
F 2 (4)
= 50.5 Table 4.3 contains the solutions of Equations 4.60
8.6
= 24.9 to 4.63 for the third stage. Table 4.4 contains the solu-
4 27.9 0 tions of Equations 4.64 to 4.66 for the second stage.
F 3 (0) Table 4.5 contains the solution of Equation 4.67 for the
= 53.4
firm 1 firm 2 firm 3
7z
first stage.
21

11
E0
From Table 4.5 we see that, given 10 units of water
available, we will obtain 53.4 net benefits and to get this
Figure 4.9. Using the forward-moving dynamic programming
method for finding the maximum accumulated net benefits, we should allocate 1 unit to Firm 1. This leaves 9 units of
Fj(Sj+1), and optimal allocations (denoted by the arrows on the water for the remaining two allocations. From Table 4.4
links) that should have been made to reach each ending state, we see that for a state of 9 units of water available we
beginning with the ending states in Stage 1, then for each ending should allocate 4 units to Firm 2. This leaves 5 units.
state in Stage 2 and finally for the ending states in Stage 3. From Table 4.3 for a state of 5 units of water available we
see we should allocate all 5 of them to Firm 3.
The arrow also tells us we should have had 9 units of Performing these calculations for various discrete total
water available at the end of Stage 1. Given this state of 9 amounts of water available, say from 0 to 38 in this
at the end of Stage 1, the arrow tells us we should allocate example, will define an allocation policy (such as the
1 unit of water to Firm 1. This is the same allocation one shown in Figure 4.6 for a different allocation
policy as obtained using the backward-moving algorithm. problem) for situations when the total amount of water is
less than that desired by all the firms. This policy can then
be simulated using alternative time series of available
4.4. Numerical Solutions
amounts of water, such as streamflows, to obtain
The application of discrete dynamic programming to estimates of the time series (or statistical measures of
most practical problems will require writing some software. those time series) of net benefits obtained by each firm,
There are no general dynamic programming computer assuming the allocation policy is followed over time.

Table 4.3. Computing the values of

E020820j

F3(S3) and optimal allocations x3 for all

remaining net benefits NB 3 ( S3 )
states S3 in Stage 3.
decisions: x 3
state S 3 4 5 6 F3 ( S3 ) x3

7 27.9 31.1 33.7 33.7 6

6 27.9 31.1 33.7 33.7 6
5 27.9 31.1 --- 31.1 5
4 27.9 --- --- 27.9 4

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 97

Optimization Methods 97

Table 4.4. Computing the values of

E020820k
F2(S2) and optimal allocations x2 for all
remaining net benefits NB 2 ( S2 ) + F3 ( S3 = S2 - x 2 )
states S2 in Stage 2.
decisions: x 2
state S 2 3 4 5 F2 ( S2 ) x2

10 15.7 + 33.7 18.6 + 33.7 21.1 + 31.1 52.3 4

9 15.7 + 33.7 18.6 + 31.1 21.1 + 27.9 49.7 4
8 15.7 + 31.1 18.6 + 27.9 --- 46.8 3

Table 4.5. Computing the values of

E020820m
F1(S1) and optimal allocations x1 for all
remaining net benefits NB 1( S1 ) + F2 ( S 2 = S1 - x1 )
states S1 in Stage 1.
decisions: x 2
state S 2 0 1 2 F2 ( S2 ) x2

10 0 + 52.3 3.7 + 49.7 6.3 + 46.8 53.4 1

4.5. Dimensionality one that proceeds forward from that node or state and
stage optimally. The forward-moving solution algorithm
One of the limitations of dynamic programming is handling is based on the principal that no matter what the state and
multiple state variables. In our water allocation example we stage (i.e., the particular node you are at), an optimal
had only one state variable: the total amount of water policy is one that has arrived at that node or state and
available. We could have enlarged this problem to include stage in an optimal manner.
other types of resources the firms require to make their This ‘principle of optimality’ is a very simple concept
products. Each of these state variables would need to be but requires the formulation of a set of recursive equa-
discretized. If, for example, only m discrete values of each tions at each stage. It also requires that either in the
state variable are considered, for n different state variables last stage (j  J) for a backward-moving algorithm, or
(e.g., types of resources) there are mn different combinations in the first stage (j  1) for a forward-moving algorithm,
of state variable values to consider at each stage. As the the future value functions, FJ1(SJ1), associated with
number of state variables increases, the number of discrete the ending state variable values, or past value functions,
combinations of state variable values increases exponen- f0(S1), associated with the beginning state variable values,
tially. This is called dynamic programming’s ‘curse of dimen- respectively, all equal some known value. Usually that
sionality’. It has motivated many researchers to search for value is 0 but not always. This condition is required in
ways of reducing the number of possible discrete states order to begin the process of solving each successive
required to find an optimal solution to large multi-state recursive equation.
problems.

4.7. Additional Applications

4.6. Principle of Optimality
Among the common dynamic programming applications
The solution of dynamic programming models or in water resources planning are water allocations to
networks is based on a principal of optimality (Bellman, multiple uses, infrastructure capacity expansion, and
1957). The backward-moving solution algorithm is based reservoir operation. The previous three-user water alloca-
on the principal that no matter what the state and stage tion problem (Figure 4.1) illustrates the first type of appli-
(i.e., the particular node you are at), an optimal policy is cation. The other two applications are presented below.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 98

98 Water Resources Systems Planning and Management

4.7.1. Capacity Expansion

How much infrastructure should be built, when and why? demand & possible capacity additions
Consider a municipality that must plan for the future
expansion of its water supply system or some component
of that system, such as a reservoir, aqueduct, or treatment demand
plant. The capacity needed at the end of each future for capacity
period t has been estimated to be Dt. The cost, Ct(st, xt), of capacity
adding capacity xt in each period t is a function of that
added capacity as well as of the existing capacity st at the
beginning of the period. The planning problem is to find

E020103a
time
that time sequence of capacity expansions that minimizes
the present value of total future costs while meeting the
predicted capacity demand requirements. This is the usual Figure 4.10. A demand projection (solid line) and a possible
capacity-expansion problem. capacity-expansion schedule for meeting that projected
This problem can be written as an optimization model: demand over time.
The objective is to minimize the present value of the total
cost of capacity expansion.
Minimize ∑Ct(st, xt) (4.101)
t
cost
where Ct(st, xt) is the present value of the cost of capacity
expansion xt in period t given an initial capacity of st.
The constraints of this model define the minimum
required final capacity in each period t, or equivalently
the next period’s initial capacity, st1, as a function of the Co
known existing capacity s1 and each expansion xt up
through period t. added capacity
E020103b

t
st1  st  ∑ xτ for t  1, 2, … , T (4.102)
τ1
Alternatively these equations may be expressed by a series Figure 4.11. Typical cost function for additional capacity given
of continuity relationships: an existing capacity. The cost function shows the fixed costs,
C0, required if additional capacity is to be added, and the
st1  st  xt for t  1, 2, … , T (4.103) economies of scale associated with the concave portion of the
In this problem, the constraints must also ensure that the cost function.

actual capacity st1 at the end of each future period t is no

less than the capacity required Dt at the end of that period.
fixed costs and economies of scale, as illustrated in
st1  Dt for t  1, 2, … , T (4.104) Figure 4.11. Each time any capacity is added there are
fixed as well as variable costs incurred. Fixed and variable
There may also be constraints on the possible expansions
costs that show economies of scale (decreasing average
in each period defined by a set Ωt of feasible capacity
costs associated with increasing capacity additions) moti-
additions in each period t:
vate the addition of excess capacity, capacity not needed
xt ∈ Ωt (4.105) immediately but expected to be needed eventually to
Figure 4.10 illustrates this type of capacity-expansion meet an increased demand for additional capacity in the
problem. The question is how much capacity to add and future.
when. It is a significant problem for several reasons. The problem is also important because any estimates
One is that the cost functions Ct(st, xt) typically exhibit made today of future demands, costs and interest rates are

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 99

Optimization Methods 99

likely to be wrong. The future is uncertain. Its uncertain-

ties increase the further we look into the future. Capacity-

capacity additions
demand & possible
expansion planners need to consider the future if their
plans are to be cost-effective. Just how far into the future
do they need to look? And what about the uncertainty in
all future estimates? These questions will be addressed
after showing how the problem can be solved for any
fixed-planning horizon and estimates of future demands,
interest rates and costs.
The constrained optimization model defined by
Equations 4.101 to 4.105 can be restructured as a multi-
stage decision-making process and solved using either a
t =1 2 3 4 5
forward or backward-moving discrete dynamic program-

E020103c
ming solution procedure. The stages of the model will be time
the time periods t. The states will be either the capacity
st1 at the end of a stage or period t if a forward-moving
solution procedure is adopted, or the capacity st, at the Figure 4.12. Network of discrete capacity-expansion
decisions (links) that meet the projected demand.
beginning of a stage or period t if a backward-moving
solution procedure is used.
A network of possible discrete capacity states and deci- f3(s4)  min{C3(s3, x3)  f2(s3)} over all discrete x3
sions can be superimposed onto the demand projection of between 0 and s4  D2
Figure 4.10, as shown in Figure 4.12. where s3  s4  x3 (4.108)
The solid blue circles in Figure 4.12 represent possible
discrete states, St, of the system, the amounts of additional In general for all stages t between the first and last:
capacity existing at the end of each period t  1 or equiv- ft(st1)  min{Ct(st, xt)  ft1(st)} over all discrete xt
alently at the beginning of period t. between 0 and st1  Dt1
Consider first a forward-moving dynamic program-
where st  st1  xt (4.109)
ming algorithm. To implement this, define ft(st1) as the
minimum cost of achieving a capacity st1, at the end of For the last stage t  T and for the final discrete
period t. Since at the beginning of the first period t  1, state sT1  DT,
the accumulated least cost is 0, f0(s1)  0. fT(sT1)  min{CT(sT, xT)  fT1(sT)} over all discrete xT
Hence, for each final discrete state s2 in stage between 0 and DT  DT1
t  1 ranging from D1 to the maximum demand DT,
define where sT  sT1  xT (4.110)
The value of fT(sT1) is the minimum present value of the
f1(s2)  min{C1(s1, x1)} in which the discrete
total cost of meeting the demand for T time periods.
x1  s2 and s1  0 (4.106) To identify the sequence of capacity-expansion decisions
Moving to stage t  2, for the final discrete states s3 rang- that results in this minimum present value of the total
ing from D2 to DT, cost requires backtracking to collect the set of best
decisions xt for all stages t. A numerical example will
f2(s3)  min{C2(s2, x2)  f1(s2)} over all discrete x2 illustrate this.
between 0 and s3  D1
where s2  s3  x2 (4.107) A numerical example

Moving to stage t  3, for the final discrete states s4 Consider the five-period capacity-expansion problem
ranging from D3 to DT, shown in Figure 4.12. Figure 4.13 is the same network

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 100

100 Water Resources Systems Planning and Management

added capacity
25 0 0 0 0

7 6 5 4
Figure 4.13. A discrete capacity-
20 0 0 0
expansion network showing the
9 9
present value of the expansion
costs associated with each 7 10 6
feasible expansion decision. 15 0 12
Finding the best path through
27 21 15 14 20 9
the network can be done using
forward or backward-moving 8 13 10
discrete dynamic programming. 10 0

11 8

5
9

t =1 2 3 4 5

3d
time

10
2 0
E0

with the present value of the expansion costs on each capacity are added in period t  1 and 15 in period
link. The values of the states, the existing capacities, t  3.
represented by the nodes, are shown on the left vertical Now we look to the question of the uncertainty of
axis. The capacity-expansion problem is solved on Figure future demands, Dt, discounted costs, Ct(st, xt), as well as
4.14 using the forward-moving algorithm. to the fact that the planning horizon T is only 5 time
From the forward-moving solution to the dynamic periods. Of importance is just how these uncertainties
programming problem shown in Figure 4.14, the present and finite planning horizon affect our decisions. While
value of the cost of the optimal capacity-expansion sched- the model gives us a time series of future capacity-
ule is 23 units of money. Backtracking (moving left expansion decisions for the next 5 time periods, what
against the arrows) from the farthest right node, this is important to decision-makers is what additional
schedule adds 10 units of capacity in period t  1, and capacity to add now, not what capacity to add in future
15 units of capacity in period t  3. periods. Does the uncertainty of future demands and
Next consider the backward-moving algorithm costs and the 5-period planning horizon affect this first
applied to this capacity-expansion problem. The general decision, x1? This is the question to address. If the answer
recursive equation for a backward-moving solution is is no, then one can place some confidence in the value
of x1. If yes, then more study may be warranted to deter-
Ft(st)  minimum{Ct(st, xt)  Ft1(st1)} over all discrete
mine which demand and cost scenario to assume, or, if
xt from Dt  st to DT  st
applicable, how far into the future to extend the planning
for all discrete states st from Dt1 to DT (4.111) horizon.
where FT1(DT)  0 and as before each cost function is Future capacity-expansion decisions in Periods 2, 3
the discounted cost. and so on can be based on updated information and
Once again, as shown in Figure 4.15, the minimum analyses carried out closer to the time those decisions
total present value cost is 23 if 10 units of additional are to be made. At those times, the forecast demands and

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 101

Optimization Methods 101

27 24 23 23 23
25 added capacity 0 0 0 0

7 6 5 4
21 21 20 20 Figure 4.14. A capacity-
20 0 0 0
expansion example, showing the
9 9
results of a forward-moving
7 10 6 dynamic programming
15 15 algorithm. The numbers next to
15 0 12
the nodes are the minimum cost
27 21 15 14 20 9
to have reached that particular
8 13 10 state at the end of the particular
11 11
10 0 time period t.
11 8

9
5
9

0
t =1 2 3 4 5
E020723o

time

0 0 0 0 0
added capacity

25 0 0 0 0

7 6 5 4
4 4 4 4 Figure 4.15. A capacity-
20 0 0 0
expansion example, showing the
9 9
results of a backward-moving
7 10 6 dynamic programming
9 9 algorithm. The numbers next to
15 0 12
the nodes are the minimum
27 21 15 14 20 9
remaining cost to have the
8 13 10 particular capacity required at
12 12
10 0 the end of the planning horizon
11 8
given the existing capacity of the
state.
17
5
9

23
t =1 2 3 4 5
3f

time
10

20
E0

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 102

102 Water Resources Systems Planning and Management

economic cost estimates can be updated and the planning 4.7.2. Reservoir Operation
horizon extended, as necessary, to a period that again
Reservoir operators need to know how much water to
does not affect the immediate decision. Note that in
release and when. Reservoirs designed to meet demands for
the example problem shown in Figures 4.14 and 4.15, the
water supplies, recreation, hydropower, the environment
use of 4 periods instead of 5 would have resulted in the
and/or flood control need to be operated in ways that meet
same first-period decision. There is no need to extend
those demands in the most reliable and effective manner.
the analysis to 6 or more periods.
Since future inflows or storage volumes are uncertain,
To summarize: What is important to decision-makers
the challenge, of course, is to determine the best reservoir
now is what additional capacity to add now. While the
release or discharge for a variety of possible inflows and
current period’s capacity addition should be based on the
storage conditions.
best estimates of future costs, interest rates and demands,
Reservoir release policies are often defined in the form
once a solution is obtained for the capacity expansion
of what are called ‘rule curves’. Figure 4.16 illustrates a
required for this and all future periods up to some distant
rule curve for a single reservoir on the Columbia River in
time horizon, one can then ignore all but that first
the northwestern United States. It combines components
decision, x1: that is, what to add now. Then just before
of two basic types of release rules. In both of these, the
the beginning of the second period, the forecasting and
year is divided into various within-year time periods.
analysis can be redone with updated data to obtain an
There is a specified release for each value of storage in
updated solution for what capacity to add in Period 2,
each within-year time period. Usually higher storage
and so on into the future. Thus, these sequential decision-
zones are associated with higher reservoir releases. If
making dynamic programming models can be designed to
the actual storage is relatively low, then less water is
be used in a sequential decision-making process.

10 3 acre feet of storage feet elevation

6000

full pool if possible full pool if possible

control for
Figure 4.16. An example winter Stl. Hd. fishing
5000 1300 CFS max for Stl. Hd. fishing
reservoir rule curve specifying and goose nesting
the storage targets and some of
the release constraints, given
4000 max pool 1600 '
the particular current storage
1600
volume and time of year. The
release constraints also include required for flood control
3000
the minimum and maximum 1550
additional FC requirement depends on forecast
release rates and the maximum
downstream channel rate of
2000 1500
flow and depth changes that can
occur in each month. min pool 1445'
1450
1000

0 Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul

assured refill curve

operating curve for power
minimum pool for boat access and campsite use during recreation season
3g
10

20
E0

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 103

Optimization Methods 103

usually released so as to hedge against a continuing water this example let the range of storage volumes be divided
shortage or drought. into intervals of 5 storage volume units. Hence, the initial
Release rules may also specify the desired storage level storage volume, St, can assume values of 0, 5, 10, 15 and
for the time of year. The operator is to release water as 20 for all periods t.
necessary to achieve these target storage levels. Maximum For each period t, let Qt be the mean inflow, Lt(St, St1)
and minimum release constraints might also be specified the evaporation and seepage losses that depend on the
that may affect how quickly the target storage levels can storage volume in the reservoir, and Rt the release or
be met. Some rule curves define multiple target storage discharge from the reservoir. Each variable is expressed as
levels depending on hydrological (e.g., snow pack) con- volume units for the period t.
ditions in the upstream watershed, or on the forecast Storage volume continuity requires that in each period
climate conditions as affected by ENSO cycles, solar geo- t the initial active storage volume, St, plus the inflow, Qt,
magnetic activity, ocean currents and the like. (There is less the losses, Lt(St, St1), and release, Rt, equals the final
further discussion of this topic in Appendix C). storage, or equivalently the initial storage, St1, in the
Reservoir release rule curves for a year, such as that following period t  1. Hence:
shown in Figure 4.16, define a policy that does not vary St  Qt  Rt  Lt(St, St1)  St1 for each period t.
from one year to the next. The actual releases will vary,
(4.112)
however, depending on the inflows and storage volumes
that actually occur. The releases are often specified inde- To satisfy the requirement (imposed for convenience in
pendently of future inflow forecasts. They are typically this example) that each storage volume variable be a dis-
based only on existing storage volumes and within-year crete value ranging from 0 to 20 in units of 5, the releases,
periods. Rt, must be such that when Qt  Rt  Lt(St, St1) is added
Release rules are typically derived from trial and error to St the resulting value of St1 is one of the 5 discrete
simulations. To begin these simulations it is useful to have numbers between 0 and 20.
at least an approximate idea of the expected impact of Assume four within-year periods t in each year
different alternative policies on various system perform- (kept small for this illustrative example). In these four
ance measures or objectives. Policy objectives could be seasons assume the mean inflows, Qt, are 24, 12, 6 and
the maximization of expected annual net benefits from 18 respectively. Table 4.6 defines the evaporation and
downstream releases, reservoir storage volumes, hydro- seepage losses based on different discrete combinations of
electric energy and flood control, or the minimization of initial and final storage volumes for each within-year
deviations from particular release, storage volume, hydro- period t.
electric energy or flood flow targets or target ranges. Rounding these losses to the nearest integer value,
Discrete dynamic programming can be used to obtain Table 4.7 shows the net releases associated with initial
initial estimates of reservoir operating policies that meet and final storage volumes. They are computed using
these and other objectives. The results of discrete Equation 4.112.
dynamic programming can be expressed in the form The information in Table 4.7 allows us to draw a
shown in Figure 4.16. network representing each of the discrete storage volume
states (the nodes), and each of the feasible releases (the
links). This network for the four seasons t in the year is
A numerical example
illustrated in Figure 4.17.
Consider, as a simple example, a reservoir having an This reservoir-operating problem is a multistage
active storage capacity of 20 million cubic metres, or for decision-making problem. As Figure 4.17 illustrates, at
that matter any specified volume units. The active storage the beginning of any season t, the storage volume can be
volume in the reservoir can vary between 0 and 20. To in any of the five discrete states. Given that state, a release
use discrete dynamic programming, this range of possible decision is to be made. This release will depend on the
storage volumes must be divided into a set of discrete state: the initial storage volume and the mean inflow, as
values. These will be the discrete state variable values. In well as the losses that may be estimated based on the

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 104

104 Water Resources Systems Planning and Management

initial final storage initial final storage

storage 0 5 10 15 20 storage 0 5 10 15 20

inflow Q 1 = 24
0 0 0.1 0.3 0.4 0.5 0 24 19 14 8 3
period t =

period t =
5 0.1 0.2 0.4 0.6 0.8 5 29 24 18 13 8
10 0.3 0.4 0.6 0.8 1.0 10 34 29 23 18 13

releases
15 0.4 0.6 0.8 1.0 1.2 15 39 33 28 23 18

losses
20 0.6 0.8 1.0 1.2 1.4 20 43 38 33 28 23
period t = 2

inflow Q 2 = 12
period t = 2
0 0 0.5 0.7 0.8 1.0 0 12 7 1 - -
5 0.5 0.7 0.8 1.0 1.2 5 17 11 6 1 -
10 0.7 0.8 1.0 1.2 1.4 10 21 61 11 6 1

releases
15 0.8 1.0 1.2 1.4 1.6 15 26 21 16 11 5
losses

20 1.0 1.2 1.4 1.6 1.8 20 31 26 21 15 10

period t = 3

period t = 3
inflow Q 3 = 6
0 0 0.7 0.9 1.0 1.2 0 6 0 - - -
5 0.7 0.9 1.0 1.2 1.4 5 10 5 0 - -
10 0.9 1.0 1.2 1.4 1.6 10 15 10 5 0 -

releases
15 1.0 1.2 1.4 1.6 1.8 15 20 15 10 4 -
losses

20 1.2 1.4 1.6 1.8 2.0 20 25 20 14 9 4

period t = 4

period t = 4
inflow Q 4 = 18

0 0 0.1 0.2 0.3 0.4 0 18 13 8 3 -

5 0.1 0.2 0.3 0.4 0.5 5 23 18 13 8 3
10 0.2 0.3 0.4 0.5 0.6 10 28 23 18 13 7

releases
15 0.3 0.4 0.5 0.6 0.7 15 33 28 23 17 12
losses

20 0.4 0.5 0.6 0.7 0.8 20 38 33 27 22 17

E020820o
E020820n

Table 4.6. Evaporation and seepage losses based on initial Table 4.7. Discrete releases associated with initial and final
and final storage volumes for example reservoir operating storage volumes.
problem.

capacity for flood protection. Table 4.8 identifies these

initial and final storage volumes, as in Table 4.6. The different targets that are to be met, if possible, for the
release will also depend on what is to be accomplished – duration of each season t.
that is, the objectives to be satisfied. Clearly, it will not be possible to meet all these storage
For this example, assume there are various targets that volume and release targets in all four seasons, given
water users would like to achieve. Downstream water inflows of 24, 12, 6 and 18, respectively. Hence, the
users want reservoir operators to meet their flow targets. objective in this example will be to do the best one can:
Individuals who use the lake for recreation want the reser- to minimize a weighted sum of squared deviations from
voir operators to meet storage volume or storage level each of these targets. The weights reflect the relative
targets. Finally, individuals living on the downstream importance of meeting each target in each season t. Target
floodplain want the reservoir operators to provide storage deviations are squared to reflect the fact that the marginal

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 105

Optimization Methods 105

20, 24 20, 12 20, 60 20, 18 20, 24

Figure 4.17. Network

representation of the four-
season reservoir release
15, 24 15, 12 15, 60 15, 18 15, 24
problem. Given any initial
storage volume St at the
beginning of a season t, and an
expected inflow of Qt during
season t, the links indicate the
10, 24 10, 12 10, 60 10, 18 10, 24 possible release decisions
corresponding to those in
Table 4.7.

5, 24 5, 12 5, 60 5, 18 05, 24

0, 24 0, 12 0, 60 0, 18 0, 24

release: R1 R2 R3 R4
stage: t =1 t =2 t =3 t=4
possible release decision
states: S t, Q t 10
3h

20
E0

Table 4.8. Storage volume and release

E020820p

targets for the example reservoir operation

problem.
period or storage targets release target
season t TS R t , TS F t TR t

15 flood control > 10

2 20 recreation > 15

3 20 recreation > 20

4 > 15

‘losses’ associated with deviations increase with increasing During the recreation season (Periods 2 and 3),
deviations. Small deviations are not as serious as deviations below or above the recreation storage lake
larger deviations, and it is better to have numerous small volume targets are damaging. During the flood season
deviations rather than a single larger one. (Period 1), any storage volume in excess of the flood

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 106

106 Water Resources Systems Planning and Management

control storage targets of 15 reduces the flood storage Finally, the last component of Equation 4.114 defines
capacity. Deviations below that flood control target are the weighted squared deficit deviations from a release
not penalized. Flood control and recreation storage target, TRt, In this example all release weights wrt equal 1.
targets during each season t apply throughout the season, Associated with each link in Figure 4.17 is the release,
thus they apply to the initial storage St as well as to the Rt, as defined in Table 4.7. Also associated with each link
final storage St1. is the sum of weighted squared deviations, TSDt, that
The objective is to minimize the sum of total weighted result from the particular initial and final storage volumes
squared deviations, TSDt, over all seasons t from now on and the storage volume and release targets identified in
into the future: Table 4.8. They are computed using Equation 4.114, with
Minimize ∑TSDt (4.113) the releases defined in Table 4.7 and targets defined in
t Table 4.8, for each feasible combination of initial and final
where storage volumes, St and St1, for each of the four seasons
TSDt  wst[(TSRt  St)2  (TS Rt  St1)2] in a year. These computed weighted squared deviations
 wfst[(ESt)2  (ESt1)2]  wrt[DR 2t ] (4.114) for each link are shown in Table 4.9.
The goal in this example problem is to find the path
In the above equation, when t  4, the last period of the through a multi-year network – each year of which is as
year, t  1  1, is the first period in the following year. shown in Figure 4.17 – that minimizes the sum of the
Each ESt is the storage volume in excess of the flood stor- squared deviations associated with each of the path’s
age target volume, TSFt. Each DRt is the difference between links. Again, each link’s weighted squared deviations are
the actual release, Rt, and the target release TRt, when the given in Table 4.9. Of interest is the best path into the
release is less than the target. The excess storage, ESt, at future from any of the nodes or states (discrete storage
the beginning of each season t can be defined by the volumes) that the system could be in at the beginning of
constraint: any season t.
St  TS tF  ESt (4.115) These paths can be found using the backward-moving
solution procedure of discrete dynamic programming.
for periods t  1 and 2, and the deficit release, DRt,
This procedure begins at any arbitrarily selected time
during period t can be defined by the constraint:
period or season when the reservoir presumably produces
Rt  TRt  DRt (4.116) no further benefits to anyone (and it doesn’t matter when
This constraint applies for all periods t. that time is – just pick any time) and proceeds backward,
The first component of the right side of Equation from right to left one stage (i.e., one time period) at a
4.114 defines the weighted squared deviations from a time, towards the present. At each node (representing a
recreation storage target, TS Rt, at the beginning and end of discrete storage volume St and inflow Qt), we can calcu-
season t. In this example the recreation season is during late the release or final storage volume in that period that
Periods 2 and 3. The weights wst associated with the minimizes the remaining sum of weighted squared
recreation component of the objective are 1 in Periods 2 deviations for all remaining seasons. Denote this
and 3. In Periods 1 and 4 the weights wst are 0. minimum sum of weighted squared deviations for all n
The second component of Equation 4.114 is for flood remaining seasons t as F nt (St, Qt). This value is dependent
control. It defines the weighted squared deviations on the state (St, Qt), and stage, t, and the number n of
associated with storage volumes in excess of the flood remaining seasons. It is not a function of the decision Rt
control target volume, TS Ft, at the beginning and end of or St1.
the flood season, period t  1. In this example, the This minimum sum of weighted squared deviations for
weights wfst are 1 for Period 1 and 0 for Periods 2, 3 and all n remaining seasons t is equal to:
4. Note the conflict between flood control and recreation n

at the end of Period 1 or equivalently at the beginning of F nt (St, Qt)  min ∑TSDt(St, Rt, St1) over all feasible
t
Period 2. values of Rt (4.117)

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 107

Optimization Methods 107

this steady-state policy by first assuming that at some time

initial final storage all future benefits, losses or penalties, F 0t (St, Qt), will be
storage 0 5 10 15 20
0. We can begin in that season t and work backwards
TR1 > 10, TS t > 15

0 0 0 0 4 74
period t =

towards the present, moving left through the network one

5 0 0 0 0 29
season t at a time. We can continue for multiple years until
10 0 0 0 0 25
the annual policy begins repeating itself each year. In other
15 0 0 0 0 25
words, when the optimal Rt associated with a particular

TSD
20 25 25 25 25 50
state (St, Qt) is the same in two or more successive years,
and this applies for all states (St, Qt) in each season t, a
steady-state policy has probably been obtained (a more
definitive test of whether or not a steady-state policy has
period t = 2
TS 2 = 20, TR 2 > 15

0 809 689 696 --- ---

5 625 466 406 446 --- been reached will be discussed later). A steady-state policy
10 500 325 216 206 296 will occur if the inflows, Qt, and objectives, TSDt(St, Rt,
15 425 250 125 66 125 St1), remain the same from year to year. This steady-state

TSD
20 400 225 100 25 25 policy is independent of the assumption that the operation
will end at some point.
To find the steady-state operating policy for this exam-
ple problem, assume the operation ends in some distant
period t = 3
TS 3 = 20, TR 3 > 20

0 996 1025 --- --- --- year at the end of Season 4 (the right-hand side nodes in
725 675 725 --- ---
5 Figure 4.17). At the end of this season the number of
10 525 425 425 525 ---
remaining seasons, n, equals 0. The values of the remain-
15 425 275 225 306 ---
ing minimum sums of weighted squared deviations,
TSD

20 400 225 136 146 256

F 10(S1, Q1) associated with each state (S1, Q1), i.e., each
node, equal 0. Now we can begin the process of finding
the best releases Rt in each successive season, moving
backward to the beginning of stage t  4, then stage
TR 4 > 15
period t = 4

0 0 4 49 144 ---

5 0 0 4 49 144
t  3, then to t  2, and then to t  1, and then to t  4
10 0 0 0 4 64
of the preceding year, and so on, each move to the left
15 0 0 0 0 9
increasing the number n remaining seasons by one.
At each stage, or season, we can compute the release Rt
TSD

20 0 0 0 0 0
E020820q

or equivalently the final storage volume St1, that

minimizes
Table 4.9. Total sum of squared deviations, TSDt, associated
with initial and final storage volumes. These are calculated
F nt (St, Qt)  Minimum{TSDt(St, Rt, St1)
using Equations 4.114 through 4.116.  F t1
n1
(St1, Qt1)} for all 0  St  20
(4.120)
The decision-variable can be either the release, Rt, or the
where
final storage volume, St1. If the decision-variable is the
St1  St  Qt  Rt  Lt(St, St1) (4.118) release, then the constraints on that release Rt are:
and Rt  St  Qt  Lt(St, St1) (4.121)
St  K, the capacity of the reservoir (4.119)
Rt  St  Qt  Lt(St, St1)  20 (the capacity) (4.122)
The policy we want to derive is called a steady-state policy.
Such a policy assumes the reservoir will be operating for a and
relatively long time with the same objectives. We can find St1  St  Qt  Rt  Lt(St, St1) (4.123)

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 108

108 Water Resources Systems Planning and Management

If the decision-variable is the final storage volume, the number of remaining seasons n  1. The data in each
constraints on that final storage volume St1 are: table are obtained from Tables 4.7 and 4.9. The last two
0  St1  20 (4.124) columns of each table represent the best release and
final storage volume decision(s) associated with the state
St1  St  Qt  Lt(St, St1) (4.125)
(initial storage volume and inflow).
and
Note that the policy defining the release or final storage
Rt  St  Qt  St1  Lt(St, St1) (4.126) for each discrete initial storage volume in season
Note that if the decision-variable is St1 in season t, this t  3 in Table 4.12 is the same as in Table 4.16, and
decision becomes the state variable in season t  1. In similarly for season t  4 in Tables 4.13 and 4.17, and for
both cases, the storage volumes in each season are limited season t  1 in Tables 4.14 and 4.18, and finally for season
to discrete values 0, 5, 10, 15 and 20. t  2 in Tables 4.15 and 4.19. The policy differs over each
Tables 4.10 through 4.19 show the values obtained state, and over each different season, but not from year to
from solving the recursive equations for 10 successive year for any specified state and season. We have reached a
seasons or stages (2.5 years). Each table represents a stage steady-state policy. If we kept on computing the release
or season t, beginning with Table 4.10 at t  4 and the and final storage policies for preceding seasons, we would

Table 4.10. Calculation of

minimum squared deviations
period t = 4 n= Q 4 = 18
associated with various discrete
F41 ( S4 , Q4 ) = min TSD4
storage states in season t = 4
with only n = 1 season remaining TSD 4
for reservoir operation. initial
storage final storage S 1 F41 R4 S1
S4 0 5 10 15 20

0 0 4 49 144 - 0 18 0
5 0 0 4 49 144 0 18-23 0-5
10 0 0 0 4 64 0 18-28 0-10
15 0 0 0 0 9 0 17-33 0-15
20 0 0 0 0 0 0 17-38 0-20
E020827a

Table 4.11. Calculation of

minimum squared deviations period t = 3 n=2 Q3 = 6
associated with various discrete
F32 ( S3 , Q3 ) = min { TSD3 ( S3 , R 3, S 4 ) + F41 ( S4 , Q4 ) }
storage states in season t = 3
with n = 2 seasons remaining for TSD 3 + F41 ( S4 , Q4 )
reservoir operation. initial
storage final storage S 4 F32 R3 S4
S3 0 5 10 15 20

0 996 1025 - - - 996 6 0

5 725 675 725 - - 675 5 5
10 525 425 425 525 - 425 5-10 5-10
15 425 275 225 306 - 225 10 10
20 400 225 136 146 256 136 14 10
E020827b

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 109

Optimization Methods 109

Table 4.12. Calculation of minimum

squared deviations associated with
period t = 2 n=3 Q 2 = 12
various discrete storage states in
F23 ( S2 , Q2 ) = min { TSD 2 ( S2 , R2 , S3 ) + F32 ( S3 , Q3 ) }
season t = 2 with n = 3 seasons
TSD 2 + F32 ( S3 , Q3 ) remaining for reservoir operation.
initial
storage final storage S 3 F 23 R2 S3
S2 0 5 10 15 20

0 809+996 689+675 696+425 - - 1121 1 10

5 625+996 466+675 406+425 446+225 - 671 1 15
10 500+996 325+675 216+425 206+225 296+136 431 6 15
15 425+996 250+675 125+425 66+225 125+136 261 5 20
20 400+996 225+675 100+425 25+225 25+136 161 10 20
E020827c

Table 4.13. Calculation of minimum

squared deviations associated with
period t = n=4 Q 1 = 24
various discrete storage states in
F14 ( S1 , Q1 ) = min { TSD1 ( S1 , R1 , S2 ) + F23 ( S2 , Q 2 ) }
season t = 1 with n = 4 seasons
TSD1 + F23 ( S2 , Q2 ) remaining for reservoir operation.
initial
storage final storage S 2 F14 R1 S2
S1 0 5 10 15 20

0 0+1121 0+671 0+431 4+261 74+161 235 3 20

5 0+1121 0+671 0+431 0+261 29+161 190 8 20
10 0+1121 0+671 0+431 0+261 25+161 186 13 20
15 0+1121 0+671 0+431 0+261 25+161 186 18 20
20 25+1121 25+671 25+431 25+261 50+161 211 23 20
E020827d

Table 4.14. Calculation of minimum

squared deviations associated with
period t = 4 n=5 Q 4 = 18
various discrete storage states in
F45 ( S4 , Q4 ) = min { TSD4 ( S4 , R 4, S1 ) + F14 ( S1 , Q1 ) } season t = 4 with n = 5 seasons
TSD4 + F14 ( S1 , Q1 )
remaining for reservoir operation.
initial
storage final storage S1 F4 5 R4 S1
S4 0 5 10 15 20

0 0+235 4+190 49+186 144+186 - 194 13 5

5 0+235 0+190 4+186 49+186 144+211 190 13-18 5-10
10 0+235 0+190 0+186 4+186 64+211 186 18 10
15 0+235 0+190 0+186 0+186 9+211 186 17-23 10-15
20 0+235 0+190 0+186 0+186 0+211 186 22-27 10-15
E020827e

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:51 AM Page 110

110 Water Resources Systems Planning and Management

Table 4.15. Calculation of

minimum squared deviations
period t = 3 n=6 Q3 = 6
associated with various discrete
F36 ( S3 , Q3 ) = min { TSD3 ( S3 , R3 , S4 ) + F45 ( S4 , Q4 ) }
storage states in season t = 3
with n = 6 seasons remaining for 5
TSD 3 + F4 ( S4 , Q4 )
reservoir operation. initial
storage final storage S4 F3 6 R3 S4
S3 0 5 10 15 20

0 996+194 1025+190 - - - 1190 6 0

5 725+194 675+190 725+186 - - 865 5 5
10 525+194 425+190 425+186 525+186 - 611 5 10
15 425+194 275+190 225+186 306+186 - 411 10 10
20 400+194 225+190 136+186 146+186 256+186 322 14 10
E020827f

Table 4.16. Calculation of

minimum squared deviations
period t = 2 n=7 Q 2 = 12
associated with various discrete
F27 ( S2 , Q2 ) = min { TSD 2 ( S2 , R 2 , S3 ) + F36 ( S3 , Q3 ) }
storage states in season t = 2
with n = 7 seasons remaining for 6
TSD 2 + F3 ( S3 , Q3 )
reservoir operation. initial
storage final storage S 3 F2 7 R2 S3
S2 0 5 10 15 20

0 809+1190 689+865 696+611 - - 1307 1 10

5 625+1190 466+865 406+611 446+411 - 857 1 15
10 500+1190 325+865 216+611 206+411 296+322 617 6 15
15 425+1190 250+865 125+611 66+411 125+322 447 5 20
20 400+1190 225+865 100+611 25+411 25+322 347 10 20
E020827g

Table 4.17. Calculation of

minimum squared deviations
period t = n=8 Q 2 = 24
associated with various discrete
F18 ( S1 , Q1 ) = min { TSD1 ( S1 , R1 , S 2 ) + F27 ( S2 , Q2 ) }
storage states in season t = 1
with n = 8 seasons remaining for TSD1 + F27 ( S 2 , Q2 )
reservoir operation. initial
storage final storage S 2 F18 R1 S2
S1 0 5 10 15 20

0 0+1307 0+857 0+617 4+447 74+347 421 3 20

5 0+1307 0+857 0+617 0+447 29+347 376 8 20
10 0+1307 0+857 0+617 0+447 25+347 372 13 20
15 0+1307 0+857 0+617 0+447 25+347 372 18 20
20 25+1307 25+857 25+617 25+447 50+347 397 23 20
E020827h

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 111

Optimization Methods 111

Table 4.18. Calculation of minimum

squared deviations associated with
period t = 4 n=9 Q 4 = 18
various discrete storage states in
F49 ( S4 , Q4 ) = min { TSD4 ( S4 , R 4, S1 ) + F18 ( S1 , Q1 ) }
season t = 4 with n = 9 seasons
TSD4 + F18 ( S1 , Q1 ) remaining for reservoir operation.
initial
storage final storage S 1 F 49 R4 S1
S4 0 5 10 15 20

0 0+421 4+376 49+372 144+372 - 380 13 5

5 0+421 0+376 4+372 49+372 144+397 376 13-18 5-10
10 0+421 0+376 0+372 4+372 64+397 372 18 10
15 0+421 0+376 0+372 0+372 9+397 372 17-23 10-15
20 0+421 0+376 0+372 0+372 0+397 372 22-27 10-15
E020827k

Table 4.19. Calculation of minimum

squared deviations associated with
period t = 3 n = 10 Q3 = 6
various discrete storage states in
F310 ( S 3 , Q3 ) = min { TSD3 ( S3 , R 3 , S 4 ) + F49 ( S4 , Q4 ) }
season t = 3 with n = 10 seasons
TSD 3 + F49 ( S4 , Q4 ) remaining for reservoir operation.
initial
storage final storage S 4 F310 R3 S4
S3 0 5 10 15 20

0 996+380 1025+376 - - - 1376 6 0

5 725+380 675+376 725+372 - - 1051 5 5
10 525+380 425+376 425+372 525+372 - 797 5 10
15 425+380 275+376 225+372 306+372 - 597 10 10
20 400+380 225+376 136+372 146+372 256+372 508 14 10
E020827m

get the same policy as that found for the same season in the conflicting, storage and release targets. There are other
following year. The policy is dependent on the state – the performance criteria that may also be evaluated using
initial storage volume in this case – and on the season t, simulation, such as measures of reliability, resilience, and
but not on the year. This policy as defined in Tables vulnerability (Chapter 10).
4.16– 4.19 is summarized in Table 4.20. Assuming the inflows that were used to derive this
This policy can be defined as a rule curve, as shown in policy actually occurred each year, we can simulate the
Figure 4.18. It provides a first approximation of a reser- derived sequential steady-state policy to find the storage
voir release rule curve that one can improve upon using volumes and releases that would occur in each period,
simulation. year after year, once a repetitive steady-state condition
Table 4.20 and Figure 4.18 define a policy that can were reached. This is done in Table 4.21 for an arbitrary
be implemented for any initial storage volume condition initial storage volume of 20 in season t  1. You can try
at the beginning of any season t. This can be simulated other initial conditions to verify that it requires only two
under different flow patterns to determine just how well years at most to reach a repetitive steady-state policy.
it satisfies the overall objective of minimizing the As shown in Table 4.21, if the inflows were repetitive
weighted sum of squared deviations from desired, but and the optimal policy was followed, the initial storage

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 112

112 Water Resources Systems Planning and Management

Table 4.20. The discrete steady-

state reservoir operating policy
release final storage volume
as computed for this example initial
season t season t
problem in Tables 4.16 to 4.19. storage
S 2 3 4 2 3 4

0 3 1 6 13 20 10 0 5

5 8 1 5 13-18 20 15 5 5-10
10 13 6 5 18 15 10 10 10
15 18 5 10 17-23 20 20 10 10-15
20 23 10 14 22-27 20 20 10 10-15
E020827n

Note from Tables 4.12 – 4.15 and 4.16 – 4.19 that

storage volume

20
27
once the steady-state sequential policy has been reached
23 10 14
22
for any specified storage volume, St, and season t, the
18
15
5
annual difference of the accumulated minimum sum of
10 squared deviations, F nt (St, Qt), equals a constant, namely
10 13 6 5
18 the annual value of the objective function. In this case
that constant is 186.
5 8 1 5 F tn4(St, Qt)  F nt (St, Qt)  186 for all St, Qt and t.
13
3 1 6 (4.127)
0
1 2 3 4 This condition indicates a steady-state policy has been
storage zone release volume season of year t
expected storage volume
achieved.
max storage target for flood control (t=1) This policy in Table 4.21 applies only for the assumed
3j
10

full reservoir storage target for recreation ( t = 2, 3 ) E020

inflows in each season. It does not define what to do if
the initial storage volumes or inflows differ from those for
Figure 4.18. Reservoir rule curve based on policy defined in which the policy is defined. Initial storage volumes and
Table 4.20. Each season is divided into storage volume zones.
inflows can and will vary from those specified in the
The releases associated with each storage volume zone are
specified. Also shown are the storage volumes that would solution of any deterministic model. One fact is certain:
result if in each year the actual inflows equalled the inflows no matter what inflows are assumed in any model, the
used to derive this rule curve. actual inflows will always differ. Hence, a policy as
defined in Table 4.20 and Figure 4.18 is much more
useful than that in Table 4.21. But as for any output from
volumes and releases would begin to repeat themselves a relatively simple optimization model, this policy should
once a steady-state condition has been reached. Once be simulated, evaluated and further refined in an effort to
reached, the storage volumes and releases will be the identify the policy that best meets the operating policy
same each year (since the inflows are the same). These objectives.
storage volumes are denoted as a blue line on the rule
curve shown in Figure 4.18. The annual total squared
4.8. General Comments on Dynamic
deviations will also be the same each year. As seen in
Programming
Table 4.21, this annual minimum weighted sum of
squared deviations for this example equals 186. This is Before ending this discussion of dynamic programming
what would be observed if the inflows assumed for this methods and its applicability for analysing water
analysis repeated themselves. resources planning problems, we should examine a major

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 113

Optimization Methods 113

problems having both unknown design and operating

year season policy decision-variables. While there are some tricks that
t S t + Qt - Rt - Lt = S t+1 TSD t
may allow dynamic programming to be used to find the
20 24 23 1 20
best solutions to both design and operating problems
2 20 12 10 2 20
encountered in water resources planning and manage-
3 20 6 14 2 10
ment studies, other optimization methods, perhaps
4 10 18 18 0 10
combined with dynamic programming where appropri-
ate, are often more useful.

year season
t S t + Qt - Rt - Lt = S t+1 TSD t 5. Linear Programming
2 10 24 14 0 20 25
2 2 20 12 10 2 20 25
If the objective function and constraints of an optimiza-
2 3 20 6 14 2 10 136
tion model are all linear, there are many readily available
2 4 10 18 18 0 10 0
computer programs one can use to find their solutions.
These programs are very powerful, and unlike many other
optimization methods, they can be applied successfully to
very large optimization problems. Many water resources
year season
t S t + Qt - Rt - Lt = S t+1 TSD t problems contain many variables and constraints, too
3 10 etc ... repeating ... many to be easily solved using non-linear or dynamic
programming methods. Linear programming procedures
E020827p

total sum of squared deviations, TSD, = 186

or algorithms for solving linear optimization models are

often the most efficient ways to find solutions to such
Table 4.21. A simulation of the derived operating policy in
problems.
Table 4.20. The storage volumes and releases in each period
t will repeat themselves each year, after the first year. The Because of the availability of computer programs that
annual total squared deviations, TSDt for the specific initial can solve linear programming problems, linear program-
and final storage volumes and release conditions are obtained ming is arguably the most popular and commonly applied
from Table 4.9. optimization algorithm. It is used to identify and evaluate
alternative plans, designs and management policies in
assumption that has been made in each of the applica- agriculture, business, commerce, education, engineering,
tions presented. The first is that the net benefits or costs finance, the civil and military branches of government,
or other objective values resulting from each decision at and many other fields.
each stage of the problem are dependent only on the state Many models of complex water resources systems are,
variable values in each stage. They are independent of or can be made, linear. Many are also very large. The
decisions made at other stages. If the returns at any stage number of variables and constraints simply defining mass
are dependent on the decisions made at other stages in a balances and capacity limitations alone at many river
way not captured by the state variables. For example, if basin sites and for numerous time periods can become
the release and storage targets or the reservoir capacity so big as to preclude the practical use of most other
were unknown in the previous reservoir operating policy optimization methods. Because of the power and
problem, then dynamic programming, with some excep- availability of computer programs that can solve large
tions, becomes much more difficult to apply. Dynamic linear programming problems, a variety of methods have
programming models can be applied to design problems, been developed to approximate non-linear (especially
such as the capacity-expansion problem or a reservoir separable) functions with linear ones just so linear
storage capacity–yield relationship as will be discussed programming can be used to solve various otherwise non-
later, or to operating problems, such as the water linear problems. Some of these methods will be described
allocation and reservoir operation problems, but rarely to shortly.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 114

114 Water Resources Systems Planning and Management

In spite of its power and popularity, for most real- 5.1. Reservoir Storage Capacity–Yield Models
world water resources planning and management prob-
lems, linear programming, like the other optimization Linear programming can be used to define storage
methods already discussed in this chapter, is best viewed capacity–yield functions for a single or multiple reser-
as a preliminary screening tool. Its value is more for voirs. A storage capacity–yield function defines the
reducing the number of alternatives for further more maximum constant ‘dependable’ reservoir release or yield
detailed simulations than for finding the best decision. that will be available, at a given level of reliability, during
This is not just because approximation methods may have each period of operation, as a function of the active
been used to convert non-linear functions to linear ones, storage volume capacity. The yield from any reservoir or
but more likely because it is difficult to incorporate all the group of reservoirs will depend on the active storage
complexity of the system and all the objectives considered capacity of each reservoir and the water that flows into
important to all stakeholders into the linear model. each reservoir, its inflow. Figure 4.19 illustrates two
Nevertheless, linear programming, like other optimiza- typical storage–yield functions for a single reservoir.
tion methods, can provide initial designs and operating To describe what a yield is and how it can be
policy information that simulation models require before increased, consider a sequence of 5 annual flows, say 2, 4,
they can simulate those designs and operating policies. 1, 5 and 3, at a site in an unregulated stream. Based on
Equations 4.45 and 4.46 define the general structure this admittedly very limited record of flows, the minimum
of any constrained optimization problem. If the objective (historically) ‘dependable’ annual flow yield of the stream
function F(X) of the vector X of decision-variables xj is linear at that site is 1, the minimum observed flow. Assuming
and if all the constraints gi(X) in Equation 4.46 are linear, the flow record is representative of what future flows
then the model becomes a linear programming model. The might be, a discharge of 1 can be ‘guaranteed’ in each of
general structure of a linear programming model is: non-zero the five time-periods of record. (In reality, that
n or any non-zero yield will have a reliability less than 1, as
Maximize or minimize ∑ Pj xj (4.128) will be considered in Chapter 11).
n j
If a reservoir having an active storage capacity of 1 is
Subject to: ∑ aij xj  bi for i  1, 2, 3, … , m (4.129) built, it could store 1 volume unit of flow when the flow
j
xj  0 for all j  1, 2, 3, … , n. (4.130) is equal to or greater than 2, and then release it along with
the natural flow when the natural flow is 1, increasing the
If any model fits this general form, where the constraints
can be any combination of equalities () and inequalities
( or ), then a large variety of linear programming
computer programs can be used to find the ‘optimal’
values of all the unknown decision-variables xj. With
reservoir yield

some exceptions, variable non-negativity is enforced

within the solution algorithms of most commercial linear lower
programming programs, eliminating the need to have to
specify these conditions in any particular application. higher
Users of linear programming need to know how to reliability
construct linear models and how to use the computer
programs that are available for solving them. They do not
E020103m

have to understand all the mathematical details of the active storage capacity of a reservoir
solution procedure incorporated in the linear program-
ming codes. But users of linear programming computer
programs should understand what the solution procedure
Figure 4.19. Two storage–yield functions for a single
does and what the computer program output means. reservoir defining the maximum minimum dependable
To begin this discussion of these topics, consider some release. These functions can be defined for varying levels of
simple examples of linear programming models. yield reliability.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 115

Optimization Methods 115

minimum dependable flow to 2 units in each year. minus the unknown yield Y and excess release, Rt, if any, in
Storing 2 units when the flow is 5, releasing 1 and the period t. (Losses are being ignored in this example.)
natural flow when that natural flow is 2, and storing 1 St  Qt  Y  Rt  St1
when the flow is 4, and then releasing the stored 2 units
for each period t  1, 2, 3, … , T. T1  1 (4.132)
along with the natural flow when the natural flow is 1,
will permit a yield of 3 in each time period with 2 units If, as indicated in Equation 4.132, one assumes that
of active capacity. This is the maximum annual yield that Period 1 follows the last Period T, it is not necessary to
is possible at this site, again based on these five annual specify the value of the initial storage volume S1 and/or
inflows and their sequence. The maximum annual yield final storage volume ST1. They are set equal to each other
cannot exceed the mean annual flow, which in this and that variable value remains unknown. The resulting
example is 3. Hence, the storage capacity–yield function ‘steady-state’ solution is based on the inflow sequence that
equals 1 when the active capacity is 0, 2 when the active is assumed to repeat itself.
capacity is 1, and 3 when the active capacity is 2. The The second set of required constraints ensures that the
annual yield remains at 3 for any active storage capacity reservoir storage volumes St at the beginning of each period
in excess of 2. t are no greater than the active reservoir capacity K.
This storage–yield function is dependent not only on St  K t  l, 2, 3, … , T (4.133)
the natural unregulated annual flows but also on their
sequence. For example if the sequence of the same 5 To derive a storage–yield function, the model defined by
annual flows were 5, 2, 1, 3, 4, the needed active storage Equations 4.131, 4.132 and 4.133 must be solved for
capacity is 3 instead of 2 volume units as before to obtain various assumed values of capacity K. Only the inflow
a dependable flow or yield of 3 volume units. In spite values Qt and reservoir active storage capacity K are
of these limitations of storage capacity–yield functions, known. All other storage, release and yield variables are
historical records are still typically used to derive them. unknown. Linear programming will be able to find their
(Ways of augmenting the historical flow record are optimal values. Clearly, the upper bound on the yield
discussed in Chapter 7.) regardless of reservoir capacity will equal the mean inflow
There are many methods available for deriving (less any losses if they were included).
storage–yield functions. One very versatile method, Alternatively, one can solve a number of linear
especially for multiple reservoir systems, uses linear programming models that minimize an unknown storage
programming. Others are discussed in Chapter 11. capacity K needed to achieve various specified yields Y.
To illustrate a storage capacity–yield model, consider a The resulting storage–yield function will be same; the
single reservoir that must provide a minimum release minimum capacity needed to achieve a specified yield will
or yield Y in each period t. Assume a record of known be the same as the maximum yield obtainable from the
(historical or synthetic) streamflows at the reservoir site corresponding specified capacity K. However, the speci-
is available. The problem is to find the maximum uniform fied yield Y cannot exceed the mean inflow. If it does,
yield Y obtainable from a given active storage capacity. there will be no feasible solution to the linear program-
The objective is to ming model.
Box 4.1 illustrates an example storage–yield model
maximize Y (4.131) and its solution to find the storage–yield function. For
This maximum yield is constrained by the water available this problem, and others in this chapter, the program
in each period, and by the reservoir capacity. Two sets of LINGO (www.lindo.com) is used.
constraints are needed to define the relationships Before moving to another application of linear
among the inflows, the reservoir storage volumes, the programming, consider how this storage–yield problem,
yields, any excess release, and the reservoir capacity. The Equations 4.131–4.133, can be formulated as a discrete
first set of continuity equations equate the unknown final dynamic programming model. The use of discrete
reservoir storage volume St1 in period t to the unknown dynamic programming is clearly not the most efficient
initial reservoir storage volume St plus the known inflow Qt, way to define a storage–yield function but the problem of

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 116

116 Water Resources Systems Planning and Management

Box 4.1. Example storage capacity–yield model

and its solution from LINGO

E020903a
! Reservoir Storage-Yield Model:
Define St as the initial active res. storage, period t,
Y as the reliable yield in each period t,
Rt as the excess release from the res., period t,
Qt as the known inflow volume to the res., period t
K as the reservoir active storage volume capacity.
;
Max = Y ; !Applies to Model 1. Must be omitted for Model 2;
Min = K ; !Applies to Model 2. Must be omitted for Model 1;
!
Subject to:
Mass balance constraints for each of 5 periods t.
;
S1 + Q1 - Y - R1 = S2;
S2 + Q2 - Y - R2 = S3;
S3 + Q3 - Y - R3 = S4;
S4 + Q4 - Y - R4 = S5;
S5 + Q5 - Y - R5 = S1; ! assumes a steady -state condition;
!
Capacity constraints on storage volumes.
;
S1 < K; S2 < K; S3 < K; S4 < K; S5 < K;
Data:
Q1 = 10; Q2 = 5; Q3 = 30; Q4 = 20; Q5 = 15;
!Note mean = 16;
K = ? ; ! Use for Model 1 only. Allows user to enter
any value of K during model run.;
Y = ? ; ! Use for Model 2 only. Allows user to enter
any value of Y during model run.
Enddata

model solutions for specified values of K in model 1 or values of Y in model

: 2

K Y S1 S2 S3 S4 S5 R1 R2 R3 R4 R5

0 5 0 0 0 0 0 5 0 25 15 10
5 10 5 5 0 5 5 0 0 15 10 5
10 12.5 10 7.5 0 2.5 10 0 0 15 0 2.5
15 15 10 10 0 15 15 0 0 0 5 0
18 16 17 11 0 14 18 0 0 0 0 0

finding a storage–yield function provides a good exercise a year with n periods remaining of reservoir operation.
in dynamic programming. The dynamic programming For initial conditions, assume all values of F 0t (St) for some
network has the same form as shown in Figure 4.19, final period t with no more periods n remaining equal a
where each node is a discrete storage and inflow state, and large number that exceeds the mean annual inflow. Then
the links represent releases. Let F nt (St) be the maximum for the set of feasible discrete total releases Rt:
yield obtained given a storage volume of St in period t of F nt (St)  max{min[Rt, F t1
n1
(St1)]} (4.134)

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 117

Optimization Methods 117

This applies for all discrete storage volumes St and for accurate they must be for the decisions that are being
all within-year periods t and remaining periods n. The considered.
constraints on the decision-variables Rt are: The stream shown in Figure 4.20 receives wastewater
effluent from two point sources located at Sites 1 and 2.
Rt  St  Qt,
Without some wastewater treatment at these sites, the
Rt  St  Qt  K, and concentration of some pollutant, Pj mg/l, at sites j  2
St1  St  Qt – Rt (4.135) and 3, will continue to exceed the maximum desired
concentration Pjmax. The problem is to find the level of
These recursive Equations 4.134 together with constraint
wastewater treatment (waste removed) at sites i  1 and
Equations 4.135 can be solved, beginning with n  1
2 that will achieve the desired concentrations at sites
and then for successive values of seasons t and remaining
j  2 and 3 at a minimum total cost.
periods n, until a steady-state solution is obtained, that
This is the classic water quality management problem
is, until
that is frequently found in the literature, although least-
F tn(St)  F tnT(St)  0 for all values of St and periods t. cost objectives have not been applied in practice. There
(4.136) are valid reasons for this that we will review later.
Nevertheless, this particular problem can serve to
The steady-state yields Ft(St) will depend on the storage illustrate the development of some linear models for
volumes St. High initial storage volumes will result in determining data needs, estimating the values of model
higher yields than will lower ones. The highest yield will parameters, and for finding, in this case, cost-effective
be that associated with the highest storage volumes and it treatment efficiencies. This problem can also serve to
will equal the same value obtained from either of the two illustrate graphically the general mathematical procedures
linear programming models. used for solving linear programming problems.
The first step is to develop a model that predicts the
pollutant concentrations in the stream as a function of
5.2. A Water Quality Management Problem
the pollutants discharged into it. To do this we need some
Some linear programming modelling and solution notation. Define Pj to be the pollutant concentration in
techniques can be demonstrated using the simple water the stream at site j. The total mass per unit time of the
quality management example shown in Figure 4.20. In pollutant (M/T) in the stream at site j will be its
addition, this example can serve to illustrate how models concentration Pj (M/L3) times the streamflow Qj (L3/T).
can help identify just what data are needed and how For example if the concentration is in units of mg/l and

firm 1 recreation
producing W1 park

Figure 4.20. A stream pollution

mflow site 2 problem that requires finding
strea
Q site W2 (1 - x 2 ) site 3 the waste removal efficiencies
W1 (1 - x 1 ) (x1, x2) of wastewater treatment
at Sites 1 and 2 that meet the
stream quality standards at
firm 2 Sites 2 and 3 at minimum total
producing W 2 cost. W1 and W2 are the amounts
of pollutant prior to treatment at
3n
10

20 Sites 1 and 2.
E0

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 118

118 Water Resources Systems Planning and Management

the flow is in terms of m3/s, and mass is to be expressed pollutant concentrations will be higher in low-flow
as kg/day: conditions than in higher-flow conditions because of
less dilution. While low-flow conditions may not provide
Mass at site j (kg/day)  Pj (mg/l) Qj (m3/s) (103 litres/m3)
as much dilution, they do result in longer travel times,
(kg/106 mg) (86400 sec./day)  86.4 PjQj (4.137)
and hence greater reductions in pollutant masses between
Each unit of a degradable pollutant mass in the stream water quality monitoring sites. Hence the pollutant
at Site 1 in this example will decrease as it travels concentrations may well be greater at some downstream
downstream to Site 2. Similarly each unit of the pollutant site when the flow conditions are higher than those of
mass in the stream at Site 2 will decrease as it travels the low-flow design value.
downstream to Site 3. The fraction a12 of the mass at In any event, given particular design streamflow and
Site 1 that reaches Site 2 is often assumed to be: temperature conditions, our first job is to determine the
values of these dimensionless transfer coefficients aij.
a12  exp(kt12) (4.138) They will be independent of the amount of waste
where k is a rate constant (1/time unit) that depends on discharged into the stream. To determine both a12 and a23
the pollutant and the temperature, and t12 is the time in this example problem (Figure 4.20) requires a number
(number of time units) it takes a particle of pollutant to of pollutant concentration measurements at Sites 1, 2 and
flow from Site 1 to Site 2. A similar expression, a23, 3 during design streamflow conditions. These measure-
applies for the fraction of pollutant mass at Site 2 that ments of pollutant concentrations must be made just
reaches Site 3. The actual concentration at the down- downstream of the wastewater effluent discharge at Site 1,
stream end of a reach will depend on the streamflow at just upstream and downstream of the wastewater effluent
that site as well as on the initial pollutant mass, the time discharge at Site 2, and at Site 3.
of travel and rate constant k. Assuming no change in streamflow and no extra
In this example problem, the fraction of pollutant pollutant entering the reach that begins just downstream
mass at Site 1 that reaches Site 3 is the product of the of Site 1 and ends just upstream of Site 2, the pollutant
transfer coefficients a12 and a23: concentration P2 just upstream of Site 2 will equal the
concentration just downstream of Site 1, P1, times the
a13  a12a23 (4.139) transfer coefficient a12:
In general, for any site k between sites i and j: P2  P1a12 (4.141)
aij  aikakj (4.140)
Under similar equal flow conditions in the following
Knowing the aij values for any pollutant and the time reach beginning just downstream from Site 2 and extend-
of flow tij permits the determination of the rate constant ing to Site 3, the pollutant concentration P3 will equal
k for that pollutant and reach, or contiguous series the concentration just downstream of Site 2, P 2, times
of reaches, from sites i to j, using Equation 4.138. If the transfer coefficient a23.
the value of k is 0, the pollutant is called a conservative If the streamflows Qi and Qj at sites i and j differ, then
pollutant; salt is an example of this. Only increased dilution the downstream pollutant concentration Pj resulting from
by less polluted water will reduce its concentration. an upstream concentration of Pi will be:
For the purposes of determining wastewater treatment
Pj  Piaij(Qi/Qj) (4.142)
efficiencies or other capital investments in infrastructure
designed to control the pollutant concentrations in the
stream, some ‘design’ streamflow conditions have to be
5.2.1. Model Calibration
established. Usually the design streamflow condition is
set at some very low flow value (e.g., the lowest monthly Sample measurements are needed to estimate the
average flow expected once in twenty years, or the values of each reach’s pollutant transport coefficients aij.
minimum seven-day average flow expected once in ten Assume five pairs of sample pollutant concentration
years). Low design flows are based on the assumption that measurements have been taken in the two stream reaches

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 119

Optimization Methods 119

during design flow conditions. For this example, also illustrating the solution of this model for the stream reach
assume that the design streamflow just downstream of between Site 1 and just upstream of Site 2 is presented in
Site 1 and just upstream of Site 2 are the same and equal Box 4.2. Again, the program LINGO (www.lindo.com) is
to 12 m3/s. The concentration samples taken just down- used to solve the models.
stream from Site 1 and just upstream of Site 2 during this Box 4.3 contains the model and solution for the
design flow condition can be used to solve Equation reach beginning just downstream of Site 2 to Site 3.
4.142 after adding error terms. More than one sample is In this reach the design streamflow is 12.5 m3/s due to
needed to allow for measurement errors and other ran- the addition of wastewater flow at Site 2.
dom effects such as those from wind, incomplete mixing In this example, based on the solutions for aij and
or varying wasteload discharges within a day. flows given in Boxes 4.2 and 4.3, a12  0.25, a23  0.60,
Denote the concentrations of each pair of sample and thus from Equation 4.140, a12 a23  a13  0.15.
measurements s in the first reach (just downstream of Site
1 and just upstream of Site 2) as SP1s and SP2s and their 5.2.2. Management Model
combined error as Es. Equation 4.142 becomes
Now that these parameter values aij are known, a water
SP2s  Es  SP1s a12 (Q1/Q2) (4.143)
quality management model can be developed. The water
The problem is to find the best estimates of the unknown quality management problem, illustrated in Figure 4.20,
a12. One way to do this is to define ‘best’ as those values involves finding the fractions xi of waste removal at
of a12 and all Es that minimize the sum of the absolute sites i  1 and 2 that meet the stream quality standards
values of all the error terms Es. This objective could be at the downstream Sites 2 and 3 at a minimum total cost.
written The pollutant concentration, P2, just upstream of Site 2
Minimize ∑ | Es | (4.144) that results from the pollutant concentration at Site 1
s
equals the total mass of pollutant at Site 1 times the frac-
The set of Equations 4.143 and 4.144 is an optimization
tion of that mass remaining at Site 2, a12, divided by the
model. If the absolute value signs can be removed, these
streamflow just upstream of Site 2, Q2. The total mass of
equations together with constraints that require all
pollutant at Site 1 at the wastewater discharge point is the
unknown variables to be non-negative would form a
sum of the mass just upstream of the discharge site, P1Q1,
linear programming model.
plus the mass discharged into the stream, W1(1  x1), at
The absolute value signs in Equation 4.144 can be
Site 1. The parameter W1 is the total amount of pollutant
removed by writing each error term as the difference
entering the treatment plant at Site 1. Similarly for Site 2.
between two non-negative variables, PEs  NEs. Thus for
Hence the concentration of pollutant just upstream of
each sample pair s:
Site 2 is:
Es  PEs  NEs (4.145)
P2  [P1Q1  W1(1x1)]a12/Q2 (4.147)
If any Es is negative, PEs will be 0 and  NEs will equal Es.
The terms P1 and Q1 are the pollutant concentration
The actual value of NEs is non-negative. If Es is positive, it
(M/L3) and streamflow (L3/T) just upstream of the waste-
will equal PEs, and NEs will be 0. The objective function,
water discharge outfall at Site 1. Their product is the mass
Equation 4.154, that minimizes the sum of absolute value
of pollutant at that site per unit time period (M/T).
of error terms, can now be written as one that minimizes
The pollutant concentration, P3, at Site 3 that results
the sum of the positive and negative components of Es:
from the pollutant concentration at Site 2 equals the total
Minimize ∑ (PEs  NEs) (4.146) mass of pollutant at Site 2 times the fraction a23. The total
s
Equations 4.143 and 4.145, together with objective mass of pollutant at Site 2 at the wastewater discharge point
function 4.146 and a set of measurements, SP1s and SP2s, is the sum of what is just upstream of the discharge site,
upstream and downstream of the reach between Sites 1 P2Q2, plus what is discharged into the stream, W2(1x2).
and 2 define a linear programming model that can be Hence the concentration of pollutant at Site 3 is:
solved to find the transfer coefficient a12. An example P3  [P2Q2  W2(1x2)]a23/Q3 (4.148)

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 120

120 Water Resources Systems Planning and Management

Box 4.2. Calibration of water quality model transfer

coefficient parameter a12

E020903b
! Calibration of Water Quality Model parameter a12.

Define variables:
SP1(k) = sample pollutant concentration just downstream of site 1 (mg/l).
SP2(k) = sample pollutant concentration just upstream of site 2 (mg/l).
PE(k) = positive error in pollutant conc. sample just upstream of site 2 (mg.l).
NE(k) = negative error in pollutant conc. sample just upstream of site 2 (mg.l).
Qi = streamflow at site i (i=1, 2), (m3/s).
a12 = pollutant transfer coefficient for stream reach between sites 1 and 2. ;
Sets:
Sample / 1..5 / : PE, NE, SP1, SP2 ;
Endsets
!
Objective: Minimize total sum of positive and negative errors.
;
Min = @sum( Sample: PE + NE )
;
! Subject to constraint for each sample k:
;
@For (Sample: a12 * SP1 = ( SP2 + PE - NE )* (Q2/Q1));

Data:
SP1 = 232, 256, 220, 192, 204;
SP2 = 55, 67, 53, 50, 51;
Q1 = 12; !Flow downstream of site 1; Q2 = 12; !Flow upstream of site 2;
Enddata

Solution: a12 = 0.25; Total sum of absolute values of deviations = 10.0

Equations 4.147 and 4.148 will become the predictive variables, x1, x2, P2, and P3. All variables are assumed to
portion of the water quality management model. The be non-negative.
remaining parts of the model include the restrictions on Some of the constraints of this optimization model can
the pollutant concentrations at Sites 2 and 3, and limits be combined to remove the two unknown concentration
on the range of values that each waste removal efficiency values, P2 and P3. Combining Equations 4.147 and 4.149,
xi can assume. the concentration just upstream of Site 2 must be no
Pj  Pjmax for j  2 and 3 (4.149) greater than P max
2 :

0  xi  1.0 for i  1 and 2. (4.150) [P1Q1  W1(1x1)]a12/Q2  P max (4.152)

2
Finally, the objective is to minimize the total cost of meeting
the stream quality standards P2max and P3max specified in Combining Equations 4.148 and 4.149, and using
Equations 4.149. Letting Ci(xi) represent the wastewater the fraction a13 (see Equation 4.139) to predict the
treatment cost function for site i, the objective can be written: contribution of the pollutant concentration at Site 1 on
the pollutant concentration at Site 3:
Minimize C1(x1)  C2(x2) (4.151)
The complete optimization model consists of Equations {[P1Q1  W1(1x1)]a13  [W2(1x2)]a23}/Q3  P max
3
4.147 through 4.151. There are four unknown decision (4.153)

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 121

Optimization Methods 121

Box 4.3. Calibration of water quality model transfer

coefficient parameter a23

E020903c
! Calibration of Water Quality Model parameter a23 .

Define variables:
SP2(k) = sample pollutant concentration just downstream of site 2 (mg/l).
SP3(k) = sample pollutant concentration at site 3. (mg/l).
PE(k) = positive error in pollutant conc. sample at site 3 (mg/l).
NE(k) = negative error in pollutant conc. sample at site 3 (mg/l).
Qi = streamflow at site i (i=2, 3), (m3/s).
a23 = pollutant transfer coefficient for stream reach between sites 2 and 3.
;
Sets:
Sample / 1..5 / : PE, NE, SP2, SP3 ;
Endsets
!
Objective: Minimize total sum of positive and negative errors.
;
Min = @sum( Sample: PE + NE )
;
! Subject to constraint for each sample k:
;
@For (Sample: a23 * SP2 = (SP3 + PE – NE )* (Q3/Q2) );

Data:
SP2 = 158, 180, 140, 150, 135;
SP3 = 96, 107, 82, 92, 81;
Q2 = 13; !Flow just downstream of site 2; Q3 = 13; !Flow at site 3;
Enddata

Solution: a23 = 0.60; Total sum of absolute values of deviations = 6.2

Equation 4.153 assumes that each pollutant discharged Subject to:

into the stream can be tracked downstream, independent Water quality constraint at Site 2:
of the other pollutants in the stream. Alternatively, [P1Q1  W1(1x1)]a12/Q2  P max (4.155)
2
Equation 4.148 computes the sum of all the pollutants
found at Site 2 and then uses that total mass to compute [(32)(10)  250000(1x1)/86.4]0.25/12  20
the concentration at Site 3. Both modelling approaches that when simplified is: x1  0.78
give the same results if the parameter values and cost Water quality constraint at Site 3:
functions are the same. [P1Q1  W1(1x1)]a13
To illustrate the solution of either of these models,  [W2(1x2)]a23}/Q3  P max (4.156)
3
assume the values of the parameters are as listed in
Table 4.22. Rewriting the water quality management {[(32)(10)  250000(1x1)/86.4]0.15
model defined by Equations 4.150 to 4.153 and substi-  [80000(1x2)/86.4]0.60}/13  20
tuting the parameter values in place of the parameters, that when simplified is: x1  1.28x2  1.79
and recalling that kg/day  86.4 (mg/l)(m3/s): Restrictions on fractions of waste removal:
Minimize C1(x1)  C2(x2) (4.154) 0  xi  1.0 for sites i  1 and 2 (4.157)

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 122

122 Water Resources Systems Planning and Management

Table 4.22. Parameter values selected for

E020827r
the water quality management problem
parameter unit value remark
illustrated in Figure 4.20.
Q1 m 3 /s 10 flow just upstream of site 1

flow
Q2 m 3 /s 12 flow just upstream of site 2
Q3 m 3 /s 13 flow at park

waste
W1 kg/day 250,000 pollutant mass produced at site 1
W2 kg/day 80,000 pollutant mass produced at site 2

P1 mg/l 32 concentration just upstream of site 1

pollutant
P2 mg/l 20 maximum allowable concentration upstream of 2

conc.
P3 mg/l 20 maximum allowable concentration at site 3

a 12 -- 0.25 fraction of site 1 pollutant mass at site 2

fraction
decay
a 13 -- 0.15 fraction of site 1 pollutant mass at site 3
a 23 -- 0.60 fraction of site 2 pollutant mass at site 2

determine the least-cost solution without knowing these

X2
costs? Models like the one just developed can be used to
1.5 determine just how accurate these cost functions (or any
equation 163 equation 160
model parameters) need to be for the decisions being
equation 160 considered.
1.0
While the actual cost functions are not known, their
feasible region
general form can be assumed, as shown in Figure 4.22.
0.5 Since the wasteloads produced at Site 1 are substantially
equation 162
greater than those produced at Site 2, and given similar
site, transport, labour, and material cost conditions, it
0.0
0.0 0.5 1.0 1.5 2.0 seems reasonable to assume that the cost of providing a
E020103o

X1
specified level of treatment at Site 1 would exceed (or
certainly be no less than) the cost of providing the same
Figure 4.21. Plot of the constraints of water quality specified level of treatment at Site 2. It would also seem
management model identifying those values of the unknown the marginal cost at Site 1 would be greater than, or at
(decision) variables x1 and x2 that satisfy all the constraints. least no less than, the marginal cost at Site 2 for the same
amount of treatment. The relative positions of the cost
The feasible combinations of x1 and x2 can be shown functions shown in Figure 4.22 are based on these
on a graph, as in Figure 4.21. This graph is a plot of assumptions.
each constraint, showing the boundaries of the region Rewriting the cost function, Equation 4.154, as a
of combinations of x1 and x2 that satisfy all the linear function converts the model defined by Equations
constraints. This red shaded region is called the feasible 4.150 through 4.151 into a linear programming model.
region. For this example problem, the linear programming model
To find the least-cost solution we need the cost func- can be written as:
tions C1(x1) and C2(x2) in Equations 4.151 or 4.154.
Suppose these functions are not known. Can we Minimize c1x1  c2x2 (4.158)

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 123

Optimization Methods 123

X2
total costs (?)
1.5

b c

feasible region
1.0
d
X 1 = 1.00
site a X 1 = 0.78 X 2 = 0.62
c1 X 2 = 0.79
0.5
2
site

0.0
c2 0.0 0.5 1.0 1.5 2.0

E020103q
X1

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
E020103p

treatment efficiency, xi , at site i Figure 4.23. Plots of various objective functions (dashed
lines) together with the constraints of the water quality
management model.
Figure 4.22. General form of total cost functions for
wastewater treatment efficiencies at Sites 1 and 2 in Figure
4.20. The dashed straight-line slopes c1 and c2 are the average marginal costs are equal. In this case the solution is
cost per unit (%) removal for 80% treatment. The actual approximately 80% treatment at both sites.
average costs clearly depend on the values of the waste Note this particular least-cost solution applies for
removal efficiencies x1 and x2 respectively.
any value of c1 greater than c2 (for example line ‘c’ in
Figure 4.23). If the marginal cost of 80% treatment at Site
1 is no less than the marginal cost of 80% treatment at
Subject to:
Site 2, then c1  c2 and indeed the 80% treatment effi-
x1  0.78 (4.159) ciencies will meet the stream standards for the design
x1  1.28 x2  1.79 (4.160) streamflow and wasteload conditions at a total minimum
cost. In fact, from Figure 4.23 and Equation 4.160, it is
0  xi  1.0 for i  1 and 2 (4.161)
clear that c2 has to exceed c1 by a multiple of 1.28 before
where the values of c1 and c2 depend on the values of x1 the least-cost solution changes to another solution. For
and x2 and both pairs are unknown. Even if we knew the any other assumption regarding c1 and c2, 80% treatment
values of x1 and x2 before solving the problem, in this at both sites will result in a least-cost solution to meeting
example the cost functions themselves (Figure 4.22) are the water quality standards for those design wasteload
unknown. Hence, we cannot determine the values of the and streamflow conditions.
marginal costs c1 and c2. However, we might be able to If c2 exceeds 1.28c1, as illustrated by line ‘d’, then the
judge which will likely be greater than the other for any least-cost solution would be x1  100% and x2  62%.
particular values of the decision-variables x1 and x2. Clearly, in this example the marginal cost, c1, of provid-
First, assume c1 equals c2. Let c1x1  c2x2 equal c and ing 100% wasteload removal at Site 1 will exceed the
assume c/c1  1. Thus x1  x2  1.0. This line can be plot- marginal cost, c2, of 60% removal at Site 2, and hence,
ted onto the graph in Figure 4.23, as shown by line ‘a’ in that combination of efficiencies would not be a least-cost
Figure 4.23. one. Thus we can be confident that the least-cost solution
Line ‘a’ in Figure 4.23 represents equal values for c1 is to remove 80% of the waste produced at both waste-
and c2, and the total cost, c1x1  c2x2, equal to 1. Keeping generating sites.
the slope of this line constant and moving it upward, rep- Note the least-cost wasteload removal efficiencies have
resenting increasing total costs, to line ‘b’, where it covers been determined without knowing the cost functions.
the nearest point in the feasible region, will identify the Why spend money defining these functions more pre-
least-cost combination of x1 and x2, again assuming the cisely? The answer: costs need to be known for financial

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 124

124 Water Resources Systems Planning and Management

planning, if not for economic analyses. No doubt the The following simple groundwater supply problem
actual costs of installing the least-cost treatment efficien- illustrates the application of some linearization methods
cies of 80% will have to be determined for issuing bonds commonly applied to non-linear separable functions –
or making other arrangements for paying the costs. functions of only one unknown variable.
However, knowing the least-cost removal efficiencies These methods typically increase the number of vari-
means we do not have to spend money defining the entire ables and constraints in a model. Some of these methods
cost functions Ci(xi). Estimating the construction and oper- require integer variables, or variables that can have values
ating costs of achieving just one wastewater removal effi- of only 0 or 1. There is a practical limit on the number of
ciency at each site, namely 80%, should be less expensive integer variables any linear programming software
than defining the total costs for a range of practical treat- program can handle. Hence, for large models there may
ment plant efficiencies that would be required to define be a need to perform some preliminary screening
the total cost functions, such as shown in Figure 4.22. designed to reduce the number of alternatives that should
Admittedly this example is relatively simple. It will not be considered in more detail. This example can be used to
always be possible to determine the ‘optimal’ solutions illustrate an approach to preliminary screening.
to linear programming problems, or other optimization
problems, without knowing more about the objective
5.3. A Groundwater Supply Example
function than was assumed for this example. However,
this exercise illustrates the use of modelling for purposes Consider a water-using industry that plans to obtain
other than finding good or ‘optimal’ solutions. Models can water from a groundwater aquifer. Two wellfield sites
help define the necessary precision of the data needed to have been identified. The first question is how much
find those good solutions. the water will cost, and the second, given any specified
Modelling and data collection and analysis should take amount of water delivered to the user, is how much
place simultaneously. All too often planning exercises are should come from each wellfield. This situation is illus-
divided into two stages: data collection and then analysis. trated in Figure 4.24.
Until one knows what data one will need, and how accu- Wells and pumps must be installed and operated to
rate those data must be, one should not spend money and obtain water from these two wellfields. The annual cost of
time collecting them. Conversely, model development in wellfield development will depend on the pumping
the absence of any knowledge of the availability and cost capacity of the wellfield. Assume that the annual costs
of obtaining data can lead to data requirements that are associated with various capacities QA and QB for Wellfields
costly, or even impossible, to obtain, at least in the time
available for decision-making. Data collection and model
development are activities that should be performed
simultaneously. industry
Because software is widely available to solve linear
programming programs, because these software programs Q
can solve very large problems containing thousands of
variables and constraints, and finally because there is less
chance of obtaining a local ‘non-optimal’ solution when QA QB
the problem is linear (at least in theory), there is an A B
incentive to use linear programming to solve large
optimization problems. Especially for large optimization wellfields
3r
10

problems, linear programming is often the only practical E0

20
alternative. Yet models representing particular water
resources systems may not be linear. This motivates the Figure 4.24. Schematic of a potential groundwater supply
use of methods that can approximate non-linear functions system that can serve a water using industry. The unknown
with linear ones. variables are the flows, QA and QB, from each wellfield.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 125

Optimization Methods 125

annual cost C (QA )

annual cost
40
35
30
26 Figure 4.25. Annual cost
20
functions associated with the
8 Wellfields A and B as shown in
0 Figure 4.24. The actual functions
well field A flow Q A 0 5 10 17
are shown on the left, and two
sets of piecewise linear
approximations are shown on
annual cost C (QB )

annual cost
26
the right.
20
18
15

0
well field B flow Q B 0 3 10 13

3 s
10
20
E0

A and B respectively are as shown in Figure 4.25. These 5.3.1. A Simplified Model
are non-linear functions that contain both fixed and vari-
able costs and hence are discontinuous. The fixed costs Two sets of approximations are shown in Figure 4.25.
result from the fact that some of the components required Consider first the approximations represented by the light
for wellfield development come in discrete sizes. As indi- blue dot–dash lines. These are very crude approximations –
cated in the figure, the maximum flow capacity of a single straight line for each function. In this example these
Wellfields A and B are 17 and 13, respectively. straight-line cost functions are lower bounds of the actual
In Figure 4.25, the non-linear functions on the left non-linear costs. Hence, the actual costs may be somewhat
have been approximated by piecewise linear functions on higher than those identified in the solution of a model.
the right. This is a first step in linearizing non-linear Using the blue dot–dash linear approximations in
separable functions. Increasing the number of linear Figure 4.25, the linear programming model can be
segments can reduce the difference between the piecewise written as follows:
linear approximation of the actual non-linear function Minimize CostA  CostB (4.162)
and the function itself. At the same time it will increase
the number of variables and possibly constraints. Subject to:
When approximating a non-linear function by a series CostA  8IA  [(408)/17]QA
of straight lines, model developers should consider two {linear approximation of C(QA)} (4.163)
factors. The first is just how accurate need be the approx-
imation of the actual function for the decisions that will CostB  15IB  [(2615)/13]QB
be made, and second is just how accurate is the actual (in {linear approximation of C(QB)} (4.164)
this case non-linear) function in the first place. There is IA, IB are 0, 1 integer variables (4.165)
little value in trying to eliminate relatively small errors
QA  17IA {limits QA to 17 and forces IA  1
caused by the linearization of a function when the
if QA
0} (4.166)
function itself is highly uncertain. Most cost and benefit
functions, especially those associated with future activi- QB  13IB {limits QB to 13 and forces IB  1
ties, are indeed uncertain. if QB
0} (4.167)

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 126

126 Water Resources Systems Planning and Management

Q  QA  QB {mass balance} (4.168) 5.3.2. A More Detailed Model

Q, QA, QB  0 {non-negativity of all decision variables} A more accurate representation of these cost functions
(4.169) may change these solutions for various values of Q,
although not significantly. However consider the more
Q  some specified amount from 0 to 30. (4.170)
accurate cost minimization model that includes the red
The expressions within the square brackets, [ ], above rep- solid-line piecewise linearizations shown in Figure 4.25.
resent the slopes of the dot–dash linear approximations of
the cost functions. The integer 0, 1 variables are required Minimize CostA  CostB (4.171)
to include the fixed costs in the model. Subject to:
Solving this model for various values of the water
CostA  {8IA1  [(208)/5]QA1}  {26IA2  [(3026)/
demand Q provides some interesting results. Again, they
(105)]QA2}  {35IA3  [(4035)/
are based on the dot–dash linear cost functions in Figure
(1710)]QA3} {linear approximation of C(QA)}
4.25. As Q increases from 0 to just under 6.8, all the water
(4.172)
will come from the less expensive Wellfield A. For any Q
from 6.8 to 13, Wellfield B becomes less expensive and all CostB  {15IB1  [(1815)/3]QB1}
the water will come from it. For any Q greater than the  {18IB2  [(2018)/(103)]QB2
capacity of Wellfield B of 13 but no greater than the  [(2620)/(1310)]QB3}
capacity of Wellfield A, 17, all of it will come from {linear approximation of C(QB)} (4.173)
Wellfield A. Because of the fixed costs, it is cheaper to use
QA  QA1  (5IA2  QA2)  (10IA3  QA3)
one rather than both wellfields. Beyond Q  17, the max-
{QA defined} (4.174)
imum capacity of A, water needs to come from both well-
fields. Wellfield B will pump at its capacity, 13, and the QB  QB1  (3IB2  QB2  QB3) {QB defined} (4.175)
additional water will come from Wellfield A.
IAi, IBi are 0, 1 integer variables for all segments i (4.176)
Figure 4.26 illustrates these solutions. One can under-
stand why in situations of increasing demands for Q over QA1  5IA1, QA2  (10  5)IA2, QA3  (17  10)IA3
time, capacity-expansion modelling might be useful. One {limits QAi to width of segment i and forces
would not close down a wellfield once developed, just to IAi  1 if QAi
0} (4.177)
achieve what would have been a least-cost solution if the
existing wellfield had not been developed. IA1 IA2  IA3  1 (4.178)
{limits solution to at most only one cost function
segment i}
QB1  3IB1, QB2  (103)IB2, QB3  (1310)IB2 (4.179)
{limits QBi to width of segment i and forces IBi  1
wellfield withdrawal

20

A
if QBi
0}
15
B IB1  IB2  1 (4.180)
10 {limits solution to at most only the first segment or to
5
the second and third segments of the cost function.
Note that a 0, 1 integer variable for the fixed cost of
0
0 5 10 15 20 25 30
the third segment of this function is not needed since
its slope exceeds that of the second segment. However
E020723p

total demand Q
the flow, QB3, in that segment must be bounded
using the integer 0, 1 variable, IB2, associated with the
Figure 4.26. Least-cost wellfield use given total demand Q second segment, as shown in the third of Equations
based on model defined by Equations 4.162 to 4.170. 4.179}

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 127

Optimization Methods 127

Q  QA  QB {mass balance} (4.181) For values of Q in excess of 27, Wellfield A remains at 17,
its capacity, and Wellfield B increases from 10 to 13.
Q, QA, QB  0 {non-negativity of all decision variables}
As in the previous example, this shows the effect
(4.182)
on the least-cost solution when one cost function has
Q  some specified amount from 0 to 30. (4.183) relatively lower fixed and higher variable costs compared
The solution to this model, shown in Figure 4.27, differs with another cost function having relatively higher fixed
from the solution of the simpler model, but only in the and lower variable costs.
details. Wellfield A supplies all the water for Q  4.3. For
values of Q between 4.4 and 13 all the water comes from 5.3.3. An Extended Model
Wellfield B. For values of Q in excess of 13 to 14.8, the
In this example, the simpler model (Equations 4.162 to
capacity of Wellfield B remains at 13 and Wellfield A
4.170) and the more accurate model (Equations 4.171
provides the additional amount of needed capacity over
to 4.183) provided essentially the same allocations of
13. For Q  14.9 to 17, the capacity of Wellfield B drops
wellfield capacities associated with a specified total
to 0 and the capacity of Wellfield A increases from 14.9
capacity Q. If the problem contained a larger number of
to 17. For values of Q between 17 and 18 Wellfield B
wellfields, the simpler (and smaller) model might have
provides 13, its capacity, and the capacity of A increases
been able to eliminate some of these wellfields from
from 4 to 5. For values of Q from 18.1 to 27, Wellfield B
further consideration. This would reduce the size of any
provides 10, and Wellfield A increases from 8.1 to 17.
new model that approximates the cost functions of the
remaining wellfields more accurately.
The model just described, like the capacity-expansion
model and water quality management model, is another
wellfield withdrawal

20
A example of a cost-effective model. The objective was to
15 find the least-cost way of providing a specified amount of
B
10
water to a water user. Next, consider a cost–benefit analy-
sis in which the question is just how much water
5 the user should use. To address this question we assume
the user has identified the annual benefits associated with
0
0 5 10 15 20 25 30 various amounts of water. The annual benefit function,
E020723q

total demand Q
B(Q), and its piecewise linear approximations, are shown
in Figure 4.28.
Figure 4.27. Least-cost wellfield use given total demand Q The straight, blue, dot–dash–dash linear approxima-
based on Equations 4.171 to 4.183. tion of the benefit function shown in Figure 4.28 is an
annual benefit B (Q)

annual benefit B (Q)

50
45
33 Figure 4.28. Benefit function of
25
the amount of water provided to
10 the water user. Piecewise linear
0 approximations of that function
flow Q 9 21 30 of flow are shown on the right.
-10
3t
10

20
E0

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 128

128 Water Resources Systems Planning and Management

upper bound of the benefits. Incorporating it into a Alternatively, to save having to add another integer
model that uses the dot–dash linear lower bound variable and constraint to the model, one can allow the
approximations of each cost function, as shown in Figure benefits to be negative. If the model solution shows nega-
4.25, will produce an optimistic solution. It is unlikely tive benefits for some small value of Q, then obviously
that the value of Q that is based on more accurate and the more preferred value of Q, and benefits, would be 0.
thus less optimistic benefit and cost functions will be any This more approximate trial-and-error approach is often
greater than the one identified by this simple optimistic preferred in practice, especially when a model contains
model. Furthermore, if any wellfield is not in the solution a large number of variables and constraints. This is
of this optimistic model, with some care we might be able the approach taken here.
to eliminate that wellfield from further consideration
when developing a more accurate model. 5.3.4. Piecewise Linearization Methods
Any component of a water resources system that does
not appear in the solution of a model that includes There are a number of ways of modelling the piecewise
optimistic approximations of performance measures that linear concave benefit function shown on the right side of
are to be maximized, such as benefits, or that are to be Figure 4.28. Several are defined in the next several sets
minimized, such as costs, are candidates for omission in of equations. Each method will result in the same model
any more detailed model. This is an example of the solution.
process of preliminary screening. One approach to modelling the concave benefit
The model defined by Equations 4.162 through 4.170 function is to define a new unrestricted (possibly negative-
can now be modified. Equation 4.170 is eliminated and the valued) variable. Let this variable be Benefits. When being
cost-minimization objective Equation 4.162 is replaced with: maximized this variable cannot exceed any of the linear
functions that bound the concave benefit function:
Maximize Benefits  (CostA  CostB) (4.184)
where Benefits  10  [(25  (10))/9]Q (4.186)
Benefits  10  [(45  25)/(21  9)]Q Benefits  10  [(45  25)/(219)]Q (4.187)
{linear approximation of B(Q)} (4.185)
Benefits  33  [(50  45)/(3021)]Q (4.188)
The solution of this model, Equations 4.163 to 4.169,
4.184, and 4.185 (plus the condition that the fixed Since most linear programming algorithms assume all
benefit of 10 only applies if Q
0, added because it is unknown variables are non-negative (unless otherwise
clear the benefits would be 0 with a Q of 0) indicates that specified), unrestricted variables, such as Benefits, can be
only Wellfield B needs to be developed, and at a capacity replaced by the difference between two non-negative
of 10. This would suggest that Wellfield A can be omitted variables, such as Pben – Nben. Pben will equal Benefits if
in any more detailed modelling exercise. To see if this its value is greater than 0. Otherwise –Nben will equal
assumption, in this example, is valid, consider the more Benefits. Thus in place of Benefits in the Equations 4.186
detailed model that incorporates the red, solid-line linear to 4.188, and those below, one can substitute Pben –
approximations of the cost and benefit functions shown Nben.
in Figures 4.25 and 4.28. Another modelling approach is to divide the variable
Note that the approximation of the generally concave Q into parts, qi, one for each segment i of the function.
benefit function in Figure 4.28 will result in negative These parts sum to Q. Each qi, ranges from 0 to the
values of the benefits for small values of Q. For example, width of the user-defined segment i. Thus for the
when the flow Q, is 0 the approximated benefits are –10. piecewise linear benefit function shown on the right of
Yet the actual benefits are 0 as shown in the left part of Figure 4.28:
Figure 4.28. Modelling these initial fixed benefits the q1  9 (4.189)
same way as the fixed costs have been modelled, using
another 0, 1 integer variable, would allow a more accurate q2  21  9 (4.190)
representation of the actual benefits for small values of Q. q3  30  21 (4.191)

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 129

Optimization Methods 129

and for incorporating special conditions into linear program-

ming models.
Q  q1  q2  q3 (4.192)

The linearized benefit function can now be written as the If–then–else conditions
sum over all three segments of each segment slope times
There exist a number of ways ‘if–then–else’ and ‘and’ and
the variable qi:
‘or’ conditions (that is, decision trees) can be included in
Benefits  10  [(25  10)/9]q1  [(45  25)/(21  9)]q2 linear programming models. To illustrate some of them,
 [(50  45)/(30  21)]q3 (4.193) assume X is an unknown decision-variable in a model
whose value may depend on the value of another
Since the function being maximized is concave (decreas-
unknown decision-variable Y. Assume the maximum
ing slopes as Q increases), we are assured that each qi1
value of Y would not exceed Ymax and the maximum
will be greater than 0 only if qi is at its upper limit, as
value of X would not exceed Xmax. These upper bounds
defined by constraint Equations 4.189 to 4.191.
and all the linear constraints representing ‘if–then–else’
A third method is to define unknown weights wi
conditions must not restrict the values of the original
associated with the breakpoints of the linearized function.
decision-variable Y. Four ‘if–then–else (with ‘and/or’)
The value of Q can be expressed as the sum of a weighted
conditions are presented below using additional integer
combination of segment endpoint values. Similarly, the
0.1 variables, denoted by Z. All the X, Y and Z variables
benefits associated with Q can be expressed as a weighted
in the constraints below are assumed to be unknown.
combination of the benefits evaluated at the segment
These constraints would be included in the any linear
endpoint values. The weights must also sum to 1.
programming model where the particular ‘if–then–else’
Hence, for this example:
conditions apply.
Benefits  (10)w1  25w2  45w3  50w4 (4.194) These illustrations are not unique. At the boundaries
of the ‘if’ constraints in the examples below, either of the
Q  0w1  9w2  21w3  30w4 (4.195) ‘then’ or ‘else’ conditions can apply.
1  w1  w2  w3  w4 (4.196)
a) If Y  50 then X  10, else X  15.
For this method to provide the closest approximation Define constraints:
of the original non-linear function, the solution must Y  50Z  Ymax(1  Z) where Z is a 0, 1
include no more than two non-zero weights and those integer variable.
non-zero weights must be adjacent to each other. Since Y  50(1  Z)
a concave function is to be maximized, this condition X  10Z  Xmax(1  Z)
will be met, since any other situation would yield less X  15(1  Z)
benefits. b) If Y  50 then X  Y, else X  Y.
The solution to the more detailed model defined by Define constraints:
Equations 4.184, 4.172 to 4.182, and either 4.186 to Y  50Z where Z is a 0, 1 integer variable.
4.188, 4.189 to 4.193, or 4.194 to 4.196, indicates a Y  50(1  Z)  YmaxZ
value of 10 for Q will result in the maximum net benefits. X  Y  XmaxZ
This flow is to come from Wellfield B. This more precise X  Y  Ymax(1Z)
solution is identical to the solution of the simpler model. c) If Y  20 or Y  80 then X  5, else X  10.
Clearly the simpler model could have successfully served Define constraints:
to eliminate Wellfield A from further consideration. Y  20Z1  80Z2  Ymax(1  Z1  Z2)
Y  20Z2  80(1  Z1  Z2)
Z1  Z2  1 where each Z is a 0, 1 integer variable.
5.4. A Review of Linearization Methods
X  5(Z1  (1  Z1  Z2))  XmaxZ2
This section presents a review of the piecewise lineariza- X  5(Z1  (1  Z1  Z2))
tion methods just described and some other approaches X  10Z2

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 130

130 Water Resources Systems Planning and Management

d) If 20  Y  50 or 60  Y  80, then X  5, Minimizing the absolute value of the difference

else X  10. between two unknown non-negative variables:
Define constraints:
Y  20Z1  50Z2  60Z3  80Z4 Minimize | X  Y | is equivalent to
 Ymax(1  Z1  Z2  Z3  Z4) Minimize D subject to X  Y  D; Y  X  D;
Y  20Z2  50Z3  60Z4 X, Y, D  0.
 80(1  Z1  Z2  Z3  Z4)
or Minimize (PDND)
Z1  Z2  Z3  Z4  1 where each Z is a 0, 1
integer variable. subject to: X  Y  PD  ND; PD, ND, X, Y  0.
X  5 (Z2  Z4)  Xmax*(1  Z2  Z4)
X  10* ((Z1  Z3)  (1  Z1  Z2  Z3  Z4))
Minimizing convex functions or maximizing concave
functions
Fixed costs in cost functions:

G (X)
cost

3 S3
S2
C=
2
variable
S1 slopes S s
cost / unit 1
C0= 0 a b
E020103v

X
fixed cost if X > 0

Maximize G(X)  Maximize B

E020103u

X Subject to: I1  S1X  B

I2  S2X  B
Cost  C0  CX if X
0, I3  S3X  B
 0 otherwise.
To include these fixed costs in a LP model, define variables: x1 x2 x3 x1 x2 x3
Cost  C0I  CX and constrain X  MI where M is slopes: S1 S2 S3 S1 S2 S3
the maximum value of X, and I is an unknown 0, 1 segments: 1 2 3 1 2 3
G (X)
F (X)

variable.

Minimizing the maximum or maximizing the

minimum

Let the set of variables be {X1, X2, X3, … , Xn} 0 a b 0 a b

E020103w

Minimize maximum {X1, X2, X3, … , Xn} is equivalent X X

to:
Minimize U subject to U  Xj, j  1, 2, 3, … , n. F(X)  S1x1  S2x2  S3x3;
Maximize minimum {X1, X2, X3, … , Xn} is equivalent to: G(X)  S1x1  S2x2  S3x3
Maximize L subject to L  Xj, j  1, 2, 3, … , n. X  x1  x2  x3; x1  a; x2  b  a

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 131

Optimization Methods 131

Minimizing or maximizing combined

segments: 1 2 3 1 2 3 concave–convex functions
weight:
w1 w2 w3 w4 w1 w2 w3 w4
F (X)

G (X)

C (X)
5
0 a b c 0 a b c 53
E020103x

X X
29 3

6
5

Minimize F(X)  F(0)w1  F(a)w2  F(b)w3  F(c)w4 0 4 12

x1 x2 x3
Maximize G(X)  G(0)w1  G(a)w2  G(b)w3  G(c)w4
X
Subject to: E020108a

X  0w1  aw2  bw3  cw4 ;

w1  w2  w3  w4  1

Maximize C(X)  (5z1  6x1  3x2)  (53z3  5x3)

Minimizing concave functions or maximizing Subject to: (x1  x2)  (12z3  x3)  X;
convex functions x1  4z1; x2  8z1; x3  99z3; z1  z3  1;
z1, z3  0, 1.
segments: 1 2 3
Minimize C(X)  (5z1  6x1)  (29z2  3x2  5x3)
G (X)

44 Subject to: x1  (4z2  x2  x3)  X;

S 3 =2

20
S 2 =3 x1  4z1; x2  8z2; x3  99 z2; z1  z2  1;
z1, z2  0, 1.
slopes S s
S 1=5
0
0 x1 4 x2 12 x3
E020103y

X
F (X)

3
35
Minimize G(X)  5x1  (20z2  3x2)  (44z3  2x3) 32 -2
22
0
Subject to: 5
6

x1  (4z2  x2)  (12z3  x3)  X; 0 x1 4 x2 12 x3 17 x4

zs  0 or 1 ∀s;
E020108b

x1  4z1; x2  8z2; x3  99 z3; z1  z2  z3  1.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 132

132 Water Resources Systems Planning and Management

Maximize or Minimize F(X) decision-variables and multiple constraints. Also like their
real-world counterparts, there are multiple feasible solu-
F(X)  (5z1  6x1)  (35z2  3x2)  (32z3  2x3)  22z4
tions to each of these problems. Hence, the task is to find
Subject to: the best solution, or a number of near-best solutions.
x1  (4z2  x2)  (12z3  x3)  (17z4  x4)  X; Each solution must satisfy all the constraints.
x1  4z1; x2  8z2; x3  5z3; x4  99z4; Constraints can reflect physical conditions, environ-
∑s zs  1; zs  0, 1 ∀s. mental regulations and/or social or economic targets.
Especially with respect to environmental or social condi-
tions and goals, it is often a matter of judgement to decide
what is considered an objective that is to be minimized or
maximized and what is considered a constraint that has to
C (X)

be met.
5 Except for relatively simple problems, the use of these
optimization models and methods is primarily for reduc-
17
3 ing the number of alternatives that need to be further
analysed and evaluated using simulation methods.
6 Optimization is generally used for preliminary screening
5 – eliminating inferior alternatives before more detailed
0
4 12 analyses are carried out. Presented were some approaches
x1 x2 x3
to preliminary screening involving calculus-based
X
Lagrange multiplier and non-linear programming meth-
ods, discrete dynamic programming methods, and linear
-17
programming methods. Each method has its strengths
E020108c

and limitations.
The example problems used to illustrate these model-
ling and model solution methods have been relatively
Maximize C(X)  (5z1  6x1  3x2)  (17z3  5x3) simple. However, simple applications such as these can
form the foundation of models of more complex prob-
Subject to: (x1  x2)  x3  X; z1, z3  0, 1.
lems, as will be shown in later chapters.
x1  4z1; x2  8z1; x3  99z3; z1  z3  1;

Minimize C(X)  (5z1  6x1)  (17z2  3x2  5x3)

Subject to: x1  (4z2  x2  x3)  X; z1, z2  0, 1.
7. Reference
x1  4z1; x2  12z2; x3  99z2; z1  z2  1 BELLMAN, R.E. 1957. Dynamic programming. Princeton,
N.J., Princeton University Press.

6. A Brief Review
Additional References (Further Reading)
Before proceeding to some other optimization and simu-
BASSSON, M.S.; ALLEN, R.B.; PEGRAM, G.G.S. and
lation methods in the following chapters, it may be useful
VAN ROOYEN, J.A. 1994. Probabilistic management of
to review the topics covered so far. The focus has been on
water resource and hydropower systems. Highlands Ranch,
model development as well as model solution. Several
Colo., Water Resources Publications.
types of water resources planning and management
problems have been used to illustrate model development BEROGGI, G.E.G. 1999. Decision modelling in policy man-
and solution processes. Like their real-world counter- agement: an introduction to analytic concepts. Boston, Mass.,
parts, the example problems all had multiple unknown Kluwer Academic.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005
wrm_ch04.qxd 8/31/2005 11:52 AM Page 133

Optimization Methods 133

BLANCHARD, B.S. and FABRYCKY, W.J. 1998. Systems MAJOR, D.C. and LENTON, R.L. 1979. Applied water
engineering and analysis, 3rd edn. Upper Saddle River, resources systems planning. Englewood Cliffs, N.J., Prentice
N.J., Prentice Hall. Hall.
BURAS, N. 1966. Dynamic programming and water MAYS, L.W. and TUNG, Y.-K. 1992. Hydrosystems
resources development, advances in hydroscience, Vol. 3. engineering and management. New York, McGraw-Hill.
New York, Academic Press.
NEWNAN, D.G.; LAVELLE, J.P. and ESCHENBACH, T.G.
DORFMAN, R.; JACOBY, H.D. and THOMAS, H.A. Jr. 2002. Essentials of engineering economic analysis, 2nd edn.
1972. Models for managing regional water quality. New York, Oxford University Press.
Cambridge, Mass., Harvard University Press.
RARDIN, R.L. 1998. Optimization in operations research.
ESOGBUE, A.O. (ed.). 1989 Dynamic programming for Upper Saddle River, N.J., Prentice Hall.
optimal water resources systems analysis. Englewood Cliffs,
N.J., Prentice Hall. REVELLE, C. 1999. Optimizing reservoir resources. New York,
John Wiley.
HALL, W.A. and DRACUP, J.A. 1970. Water resources sys-
tems engineering. New York, McGraw-Hill. SOMLYODY, L. 1997. Use of optimization models in
river basin water quality planning: WATERMATEX’97.
HILLIER, F.S. and LIEBERMAN, G.J. 1990. Introduction to In: M.B. Beck and P. Lessard (eds.), Systems analysis and
operations research, 5th edn. New York, McGraw-Hill. computing in water quality management: towards a new
HILLIER, F.S. and LIEBERMAN, G.J. 1990. Introduction agenda. London, IWA.
to stochastic models in operations research. New York, TAYLOR, B.W. III. 1999. Introduction to management
McGraw-Hill. science, 6th edn. Upper Saddle River, N.J., Prentice Hall.
HUFSCHMIDT, M.M. and FIERING, M.B. 1966. Simulation WILLIS, R. and YEH, W.W.-G. 1987. Groundwater
techniques for design of water-resource systems. Cambridge, systems planning and management. Englewood Cliffs, N.J.,
Mass., Harvard University Press. Prentice Hall.
KARAMOUZ, M.; ZINSSER, W.K. and SZIDAROVSZKY, F.
WINSTON, W.L. 1994. Operations research: applications
2003. Water resources systems analysis. Boca Raton, Fla.,
and algorithms, 3rd edn. Boston, Mass., Duxbury.
Lewis.
WINSTON, W.L. and ALBRIGHT, S.C. 2001. Practical
LOUCKS, D.P.; STEDINGER, J.S. and HAITH, D.A.
management science, 2nd edn. Pacific Grove, Calif., Duxbury.
1981. Water resource systems planning and analysis.
Englewood Cliffs, N.J., Prentice Hall. WRIGHT, J.R. and HOUCK, M.H. 1995. Water resources
planning and management. In: W.F. Chen (ed.), Civil
MAASS, A.; HUFSCHMIDT, M.M.; DORFMAN, R.;
engineering handbook, Chapter 37. Boca Raton, CRC.
THOMAS, H.A. Jr.; MARGLIN, S.A. and FAIR, G.M.
1962. Design of water-resource systems. Cambridge, Mass., WURBS, R.A. 1996. Modelling and analysis of reservoir
Harvard University Press. system operations. Upper Saddle River, N.J., Prentice Hall.

WATER RESOURCES SYSTEMS PLANNING AND MANAGEMENT – ISBN 92-3-103998-9 – © UNESCO 2005