Scribd
Upload
84 views15 pages

Week 4

The document discusses necessary and sufficient conditions for unconstrained and equality constrained optimization problems. It provides: 1) Definitions of necessary conditions (NC), which must be satisfied at an optimum point but don't guarantee optimality, and sufficient conditions (SC), which guarantee optimality if satisfied. 2) The conditions are different for unconstrained versus constrained problems, where in constrained problems optima must satisfy constraints. 3) For unconstrained problems, first and second order NC involve finding stationary points where the first derivative is zero and the second derivative is positive. SC require the second derivative be positive. 4) For equality constrained problems, the Lagrange multiplier theorem provides NC involving setting derivatives of the cost function

Uploaded by

Mohmmad Break
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
84 views15 pages

Week 4

The document discusses necessary and sufficient conditions for unconstrained and equality constrained optimization problems. It provides: 1) Definitions of necessary conditions (NC), which must be satisfied at an optimum point but don't guarantee optimality, and sufficient conditions (SC), which guarantee optimality if satisfied. 2) The conditions are different for unconstrained versus constrained problems, where in constrained problems optima must satisfy constraints. 3) For unconstrained problems, first and second order NC involve finding stationary points where the first derivative is zero and the second derivative is positive. SC require the second derivative be positive. 4) For equality constrained problems, the Lagrange multiplier theorem provides NC involving setting derivatives of the cost function

Uploaded by

Mohmmad Break
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 15

Arora 4.3-4.

5
– Necessary and sufficient conditions
– Unconstrained problems
– Equality constrained problems

[email protected]
1
4.3 Necessary and Sufficient
Conditions
• Necessary Conditions (NC)
– Must be satisfied at optimum point
– Do not guarantee optimality
– Points that satisfy necessary conditions are called
candidate points
– If a point does not satisfy the necessary conditions (is not a
candidate point), it cannot be the optimum
• Sufficient Conditions (SC)
– A candidate point that satisfies the sufficient conditions is
optimum
– If the sufficient conditions are not satisfied, we cannot
always conclude that the candidate point is not optimum

2
Unconstrained vs. Constrained
Optimum Design
In unconstrained optimization, every point is
feasible
In constrained optimization, minima outside of
the feasible range are not solutions

3
Unconstrained vs. Constrained Optimum Design
Unconstrained Constrained
Equality Constrained
x2 f(x1,x2)=1
x2
f(x1,x 2)=2
f(x1,x2)=3
Feasible f(x1,x2)=4
Region x1 x1

h(x1,x2)=0

Inequality constrained
f(x1,x2)=2
x2 Feasible
f(x1,x 2)=3
Region
f(x1,x2)=4

x1
4
4.4 Optimality Conditions for
Unconstrained Problems
Minimize f(x) without any constraints on x
• We use optimality conditions to
– Check whether a point is a local minimum
– Solve for locally optimal points
• To obtain local optimality conditions
– Assume x* is a local minimum
– Examine the function and its derivatives in a small
neighborhood about x*
– If we move a small distance away, f(x) will not
decrease

5
Necessary Conditions
First order necessary condition for a local minimum
(maximum) at x*
𝑓 ′ 𝒙∗ = 0
Stationary points
– Local minima
– Local maxima
– Inflection points

Second order necessary condition for a local


minimum at x*
𝑓 ′′ 𝒙∗ ≥ 0
6
Sufficient Conditions
Sufficient condition for a point that satisfies the
first order necessary condition to be a local
minimum
𝑓 ′′ 𝒙∗ > 0

Note, if 𝑓 ′′ 𝒙∗ = 0, we need to evaluate


higher-order derivatives

7
NCs and SCs
• The necessary conditions must be satisfied at
a minimum point.
• The necessary conditions may also be
satisfied at points that are not minima. A point
that satisfies the necessary conditions is
simply a candidate local minimum.
• If the sufficient condition is satisfied at a
candidate point, then it is a minimum.

8
Optimality Conditions for Functions of Several Variables

First order necessary condition


❑𝑓 𝒙∗ = 0
or
𝜕𝑓 𝒙∗
= 0 for 𝑖 = 1 𝑡𝑜 𝑛
𝜕𝑥 𝑖

Second order necessary condition


𝜕2𝑓
𝐻 𝒙∗ = is positive semidefinite or positive definite at 𝒙∗
𝜕𝑥 𝑖 𝜕𝑥 𝑗
𝑛×

Second order sufficient condition


If 𝐻 𝒙∗ is positive definite at 𝒙∗, then 𝒙∗ is a local minimum

Other
If 𝐻 𝒙∗ is indefinite at 𝒙∗, then 𝒙∗ is an inflection point

9
4.5 Necessary Conditions for Equality-
Constrained Problems
A point 𝒙∗ satisfying the constraints 𝒉(𝒙∗) = 0 is a
regular point of the feasible set if
• 𝒇(𝒙∗) is differentiable and
• gradient vectors of all active constraints at 𝒙∗ are
linearly independent
Note: If the problem includes inequality constraints,
the gradients of all active constraints must be linearly
independent

10
Lagrange Multiplier Theorem
Consider the equality constrained optimization problem

Minimize 𝑓 𝒙
Subject to ℎ𝑖 𝒙 = 0; 𝑖 = 1 𝑡𝑜 𝑝

Let 𝒙∗ be a regular point and a local minimum for the problem.


Then there exist unique Lagrange multipliers 𝑣 ∗ ; 𝑗 = 1 𝑡𝑜 𝑝
such that

𝜕𝑓 𝒙∗ 𝑝 ∗ 𝜕ℎ 𝑗 𝒙∗
+ σ 𝑗=1 𝑣𝑗 = 0; 𝑖 = 1 𝑡𝑜 𝑛
𝜕 𝜕
ℎ𝑗 𝒙∗ = 0; 𝑗 = 1 𝑡𝑜 𝑝

11
Properties of Lagrange Function

• Lagrange function is stationary with respect to 𝒙 and 𝒗, so it can be


treated as an unconstrained function in 𝒙 and 𝒗 to determine
stationary points
• Points that satisfy the necessary conditions are candidate points
(may be minimum, maximum, or inflection points)
• We need 2nd order NCs and SCs to distinguish between them
• In equality constrained problems, Lagrange multipliers are free in
sign
• The gradient of the cost function is a linear combination of the
gradients of the constraints at the candidate point, with 𝑣∗ as the
scalars of the linear combinations
• Often, NCs of the Lagrange Multiplier Theorem lead to a set of
nonlinear equations that can’t be solved analytically

12
4.4 Examples
1. Find local minima for
𝑓 𝑥 = cos(𝑥)

2. Find local min and max points for


𝑓 𝑥 = 𝑥 3 − 𝑥 2 − 4𝑥 + 4

13
4.5 Examples
1. Maximize
𝑓 𝑥1 , 𝑥2 , 𝑥3 = 𝑥1𝑥2 + 𝑥2𝑥3 + 𝑥1𝑥3
Subject to
ℎ 𝑥1, 𝑥2, 𝑥3 = 𝑥1 + 𝑥2 + 𝑥3 − 9 = 0

2. Minimize
𝑓 𝑥1, 𝑥2 = 𝑥12 − 2𝑥1𝑥2 + 3𝑥22 − 4
Subject to
ℎ 𝑥1, 𝑥2 = −𝑥1 + 𝑥2 − 2 = 0

14
Example: Minimum Cost Cylindrical
Tank Design
Design a minimum-cost cylindrical tank closed at both
ends to contain a fixed volume V. The cost depends
directly on the area of sheet metal used.
Let
C = $ per unit area of sheet metal
R = tank radius
H = tank height

15

You might also like