Scribd
Upload
50 views7 pages

Goal:: Want To Find The Maximum or Minimum of A Function Subject To Some Constraints

The document discusses optimization and the goal of finding the maximum or minimum of a function subject to constraints. It provides the formal statement of an optimization problem and outlines how sufficient and necessary conditions for local optimality will be derived/stated for problems with various constraint configurations. The key conditions discussed are the Karush-Kuhn-Tucker (KKT) conditions, which provide a test to determine if a point is a local minimum for a problem involving equality and inequality constraints.

Uploaded by

arst
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
50 views7 pages

Goal:: Want To Find The Maximum or Minimum of A Function Subject To Some Constraints

The document discusses optimization and the goal of finding the maximum or minimum of a function subject to constraints. It provides the formal statement of an optimization problem and outlines how sufficient and necessary conditions for local optimality will be derived/stated for problems with various constraint configurations. The key conditions discussed are the Karush-Kuhn-Tucker (KKT) conditions, which provide a test to determine if a point is a local minimum for a problem involving equality and inequality constraints.

Uploaded by

arst
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

Optimization

Goal:
Want to find the maximum or minimum of a function subject to
some constraints.
Formal Statement of Problem:
Given functions f , g1 , . . . , gm and h1 , . . . , hl defined on some
domain ⌦ ⇢ Rn the optimization problem has the form

min f (x) subject to gi (x)  0 8i and hj (x) = 0 8j


x2⌦
In these notes..

We will derive/state sufficient and necessary for (local) optimality


when there are
1 no constraints,
2 only equality constraints,
3 only inequality constraints,
4 equality and inequality constraints.
From this fact Lagrange Multipliers make sense

Remember our constrained optimization problem is

min f (x) subject to h(x) = 0


x2R2

Define the Lagrangian as note L(x⇤ , µ⇤ ) = f (x⇤ )


#

L(x, µ) = f (x) + µ h(x)

Then x⇤ a local minimum () there exists a unique µ⇤ s.t.


1 rx L(x⇤ , µ⇤ ) = 0 encodes rx f (x⇤ ) = µ⇤ rx h(x⇤ )

2 rµ L(x⇤ , µ⇤ ) = 0 encodes the equality constraint h(x⇤ ) = 0


3 yt (r2xx L(x⇤ , µ⇤ ))y 0 8y s.t. rx h(x⇤ )t y = 0
"
Positive definite Hessian tells us we have a local minimum
Summary of optimization with one inequality constraint

Given

min f (x) subject to g(x)  0


x2R2

If x⇤ corresponds to a constrained local minimum then


Case 1: Case 2:
Unconstrained local minimum Unconstrained local minimum
occurs in the feasible region. lies outside the feasible region.

1 g(x⇤ ) < 0 1 g(x⇤ ) = 0

2 rx f (x⇤ ) = 0 2 rx f (x⇤ ) = rx g(x⇤ )


with > 0
3 rxx f (x⇤ ) is a positive
semi-definite matrix. 3 yt rxx L(x⇤ ) y 0 for all
y orthogonal to rx g(x⇤ ).
Karush-Kuhn-Tucker conditions encode these conditions
Given the optimization problem

min f (x) subject to g(x)  0


x2R2

Define the Lagrangian as

L(x, ) = f (x) + g(x)

Then x⇤ a local minimum () there exists a unique ⇤


s.t.

1 rx L(x⇤ , ⇤
)=0
⇤
2 0
⇤
3 g(x⇤ ) = 0
4 g(x⇤ )  0
5 Plus positive definite constraints on rxx L(x⇤ , ⇤
).

These are the KKT conditions.


Let’s check what the KKT conditions imply

Case 1 - Inactive constraint:

• When ⇤ = 0 then have L(x⇤ , ⇤) = f (x⇤ ).


• Condition KKT 1 =) rx f (x⇤ ) = 0.
• Condition KKT 4 =) x⇤ is a feasible point.

Case 2 - Active constraint:

• When ⇤ > 0 then have L(x⇤ , ⇤) = f (x⇤ ) + ⇤ g(x⇤ ).

• Condition KKT 1 =) rx f (x⇤ ) = ⇤r


x g(x
⇤ ).

• Condition KKT 3 =) g(x⇤ ) = 0.


• Condition KKT 3 also =) L(x⇤ , ⇤ ) = f (x⇤ ).
KKT for multiple equality & inequality constraints
Given the constrained optimization problem

min f (x)
x2R2

subject to

hi (x) = 0 for i = 1, . . . , l and gj (x)  0 for j = 1, . . . , m

Define the Lagrangian as


t
L(x, µ, ) = f (x) + µt h(x) + g(x)
⇤
Then x⇤ a local minimum () there exists a unique s.t.
1 rx L(x⇤ , µ⇤ , ⇤
)=0
⇤
2 j 0 for j = 1, . . . , m
⇤
3 j gj (x⇤ ) = 0 for j = 1, . . . , m
4 gj (x⇤ )  0 for j = 1, . . . , m
5 h(x⇤ ) = 0
6 Plus positive definite constraints on rxx L(x⇤ , ⇤
).

You might also like