2 Nonlinear Programming Models
Topics Convex sets and convex programming First-order optimality conditions Examples Problem classes
General NLP
Minimize f(x) s.t. gi(x) (, , =) bi, i = 1,,m x = (x1,,xn)T is the n-dimensional vector of decision variables f (x) is the objective function gi(x) are the constraint functions bi are fixed known constants
Convex Sets
Definition: A set S n is convex if every point on the line segment connecting any two points x1, x2 S is also in S. Mathematically, this is equivalent to x0 = x1 + (1)x2 S for all such 0 1. x1
x1
x2
x1
x2 x2
(Nonconvex) Feasible Region
S = {(x1, x2) : (0.5x1 0.6)x2 1 2(x1)2 + 3(x2)2 27; x1, x2 0}
x2
x1
Convex Sets and Optimization
Let S = { x n : gi(x) bi, i = 1,,m } Fact: If gi(x) is a convex function for each i = 1,,m then S is a convex set. Convex Programming Theorem: Let x n and let f (x) be a convex function defined over a convex constraint set S. If a finite solution exists to the problem Minimize { f (x) : x S } then all local optima are global optima. If f (x) is strictly convex, the optimum is unique.
Convex Programming
Min f (x1,,xn) s.t. gi(x1,,xn) bi i = 1,,m x1 0,,xn 0 is a convex program if f is convex and each gi is convex. Max f (x1,,xn) s.t. gi(x1,,xn) bi i = 1,,m x1 0,,xn 0 is a convex program if f is concave and each gi is convex.
Linearly Constrained Convex Function with Unique Global Maximum
Maximize f (x) = (x1 2)2 + (x2 2)2 subject to 3x1 2x2 6 x1 + x2 3 x1 + x2 7 2x1 3x2 4
x2
5 4 3 2 1
x1
(Nonconvex) Optimization Problem
First-Order Optimality Conditions
Minimize { f (x) : gi(x) bi, i = 1,,m } Lagrangian:L(x,) = f (x) + i (gi (x) bi )
m i=1
Optimality conditions Stationarity: L(x,) = f (x) + igi (x) = 0 Complementarity: igi(x) = 0, i = 1,,m Feasibility: gi(x) bi, i = 1,,m Nonnegativity: i 0, i = 1,,m
i=1 m
Importance of Convex Programs
Commercial optimization software cannot guarantee that a solution is globally optimal to a nonconvex program. NLP algorithms try to find a point where the gradient of the Lagrangian function is zero a stationary point and complementary slackness holds. Given we want L(x,) = f(x) + (g(x) b) L(x,) = f(x) + g(x) = 0 (g(x) b) = 0 g(x) b 0, 0
For a convex program, all local solutions are global optima.
Example: Cylinder Design
We want to build a cylinder (with a top and a bottom) of maximum volume such that its surface area is no more than s units. Max V(r,h) = r2h s.t. 2r2 + 2rh = s r 0, h 0
There are a number of ways to approach this problem. One way is to solve the surface area constraint for h and substitute the result into the objective function.
r
Solution by Substitution
s 2r 2 h= 2r s 2r 2 rs Volume = V = r2 [ r 3 ] = 2 2r
s 1/2 s 1/2 dV s = 0 r=( ) h = 2r r = 2( ) dr 6 6 V = r 2h s 3/2 = 2 ( 6) s 1/2 r = ( ) 6 s 1/2 ) h = 2( 6
Is this a global optimal solution?
Test for Convexity
rs r 3 V(r ) = 2 s dV(r) 3r 2 = 2 dr d2V(r ) dr 2 = 6r
d2V 0 for all r 0 dr 2 Thus V(r ) is concave on r 0 so the solution is a global maximum.
Advertising (with Diminishing Returns)
A company wants to advertise in two regions. The marketing department says that if $x1 is spent in region 1, sales volume will be 6(x1)1/2. If $x2 is spent in region 2, sales volume will be 4(x2)1/2. The advertising budget is $100.
Model: Max f (x) = 6(x1)1/2 + 4(x2)1/2 s.t.
x1 + x2 100, x1 0, x2 0
Solution: x1* = 69.2, x2* = 30.8, f (x*) = 72.1 Is this a global optimum?
Excel Add-in Solution
A B C D 1 Nonlinear Model 2 72.111 Change 3 2 4 TRUE Solve 5 TRUE 100 Variables 6 Change Relation Name: 7 Values: 8 Lower Bounds: 9 10 11 Linear Obj. Coef.: 12 Nonlinear Obj. Terms: 13 Nonlinear Obj. Coef.: 14 Constraints 15 Num. Name Value Rel. RHS 16 1 Con1 100 <= 100 17 2 Con2 0 <= 10000 18 19 E F G H I J K L Name: Adv100 Objective Terms Solver: Excel Solver Type: NLP1 Linear: 0 Type: Nonlinear Goal: Max NonLinear 1: 72.111 Sens.: Yes 0 Objective: 72.111NonLinear 2: 1 2 X1 X2 69.231 30.769 0 0 M N O
Comp. Time 00:00 Status Optimal
0 8.3205 6
0 5.547 4
Linear Constraint Coefficients 1 1 0 0
Portfolio Selection with Risky Assets (Markowitz)
Suppose that we may invest in (up to) n stocks. Investors worry about (1) expected gain (2) risk. Let rj = random variable associated with return on stock j
j = expected return on stock j jj = variance of return for stock j
We are also concerned with the covariance terms:
ij = cov(ri, rj)
If ij > 0 then returns on i and j are positively correlated. If ij < 0 returns are negatively correlated.
Decision Variables: xj = # of shares of stock j purchased
Expected return of the portfolio: R(x) = jxj
j =1 n
Variance (measure of risk): V(x) =
11 12 1 1 = 21 22 1 1
i =1 j =1
ijxixj
11 12 1 1 = If , then buying stock 2 is just like 21 22 1 1
buying additional shares of stock 1.
Nonlinear optimization models
Let pj = price of stock j b = our total budget
= risk-aversion factor (when = 0 risk is not a factor)
Consider 3 different models: 1) Max f (x) = R(x) V(x) s.t.
j =1 n
pj xj b, xj 0, j = 1,,n
where 0 is determined by the decision maker
2) Max f (x) = R(x) s.t. V(x) , pjxj b, xj 0, j = 1,,n
j =1 n
where 0 is determined by the investor. Smaller values of represent greater risk aversion. 3) Min f (x) = V(x) s.t. R(x) , pj xj b, xj 0, j = 1,,n
j =1 n
where 0 is the desired rate of return (minimum expectation) is selected by the investor.
Hanging Chain with Rigid Links
10ft
1 ft
x
y
each link
What is equilibrium shape of chain? Decision variables: Let (xj, yj), j = 1,,n, be the incremental horizontal and vertical displacement of each link, where n 10. Constraints: xj2 + yj2 = 1, j = 1,,n, each link has length 1 x1 + x2 + y1 + y2 +
+ xn = 10, net horizontal displacement + yn = 0, net vertical displacement
Objective: Minimize chains potential energy Assuming that the center of the mass of each link is at the center of the link. This is equivalent to minimizing
1 y + (y + 1 y ) + (y + y + 1 y ) + 1 1 1 2 2 3 2 2 2
+ (y1 + y2 +
1 + yn-1 + 2 yn)
1 1 = (n 1 + 2 ]y1 + (n 2 + 2 )y2 1 + (n 3 + 2 )y3 +
3 1 + 2 yn-1 + 2 yn
Summary
Min (n j + )yj
j =1 n
s.t. xj2 + yj2 = 1, j = 1,,n x1 + x2 + y1 + y2 +
+ xn = 10 + yn = 0
Is a local optimum guaranteed to be a global optimum?
No!
Constraints xj2 + yj2 = 1 for all j yield a nonconvex feasible region so there may be several local optima. Consider a chain with 4 links:
These solutions are both local minima.
Direct Current Network
10 20
I2
I4
I3
100v
I1
I6
10
I7
I5
20
Problem: Determine the current flows I1, I2,,I7 so that the total content is minimized Content: G(I) =
v(i)di for I 0 and G(I) = I v(i)di for I < 0
Solution Approach
Electrical Engineering: Use Kirchoffs laws to find currents when power source is given. Operations Research: Optimize performance measure in network taking flow balance into account. Linear resistor: Voltage, v(I ) = IR Content function, G(I ) = I 2R/2 Battery: Voltage, v(I ) = E Content function, G(I ) = EI
Network Flow Model
Network diagram:
2
5 I2
2
10 I 4
-100 I1
5 I3
10 I5
0 I6
0 I7
Minimize Z = 100I1 + 5I2 + 5I3 + 10I4 + 10I5 subject to I1 I2 = 0, I2 I3 I4 = 0, I5 I6 = 0, I5 + I7 = 0, I3 + I6 I7 = 0, I1 I6 = 0 Solution: I1 = I2 = 50/9, I3 = 40/9, I4 = I5 = 10/9, I6 = 50/9, I7 = 10/9
NLP Problem Classes
Constrained vs. unconstrained Convex programming problem Quadratic programming problem
f (x) = a + cTx + xTQx, Q 0
Separable programming problem
f (x) = j=1,n fj(xj)
Geometric programming problem
g(x) = t=1,T ctPt(x), Pt(x) = (x1 t1) . . . (xn tn), xj >0
a a
Equality constrained problems
What You Should Know About Nonlinear Programming
How to identify a convex program. How to write out the first-order optimality conditions. The difference between a local and global solution. How to classify problems.
