Scribd
Upload
14 views1 page

Constrained Optimization Problems: Lagrangian and Lagrange Multipliers

The document discusses constrained optimization problems in a Hilbert space, focusing on the formulation of the Lagrangian and the use of Lagrange multipliers. It introduces concepts such as saddle points, strong duality, and the conditions under which a minimizer exists, particularly in the convex case. Additionally, it covers differentiable cases with the Fritz-John and KKT theorems, outlining necessary conditions for local minimizers.

Uploaded by

Rana AL ALI
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
14 views1 page

Constrained Optimization Problems: Lagrangian and Lagrange Multipliers

The document discusses constrained optimization problems in a Hilbert space, focusing on the formulation of the Lagrangian and the use of Lagrange multipliers. It introduces concepts such as saddle points, strong duality, and the conditions under which a minimizer exists, particularly in the convex case. Additionally, it covers differentiable cases with the Fritz-John and KKT theorems, outlining necessary conditions for local minimizers.

Uploaded by

Rana AL ALI
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 1

Constrained Optimization Problems

Let H be a Hilbert space and f : H →] − ∞, +∞] a proper function. Let (m, q) ∈ N2 , with gi : H → R for i = 1, ..., m and
hj : H → R for j = 1, ..., q. Define the constraint set:

C = {x ∈ H | gi (x) = 0, hj (x) ≤ 0 ∀i, j}

Goal: Find x̂ ∈ Argminx∈C f (x).


Admissibility: x ∈ H is admissible if x ∈ dom f ∩ C.

Lagrangian and Lagrange Multipliers


The Lagrangian is:
m
X q
X
L(x, ν, λ) = f (x) + νi gi (x) + λj hj (x), λ ∈ Rq+
i=1 j=1

ν and λ are Lagrange multipliers.

Lagrange Duality
• Primal Lagrangian: L(x) = supν,λ≥0 L(x, ν, λ)
• Dual Lagrangian: L∗ (ν, λ) = inf x∈H L(x, ν, λ)
• L = f + ιC , −L∗ is convex and lower semicontinuous

Weak duality: L∗ (ν, λ) ≤ L(x) for all (x, ν, λ)

sup L∗ (ν, λ) ≤ µ = inf f (x)


ν,λ x∈C

Saddle Points and Strong Duality


Saddle point: (x̂, ν̂, λ̂) is a saddle point if:

∀(x, ν, λ), L(x̂, ν, λ) ≤ L(x̂, ν̂, λ̂) ≤ L(x, ν̂, λ̂)

If (x̂, ν̂, λ̂) is a saddle point, then:


µ = f (x̂) = L∗ (ν̂, λ̂), and λ̂j hj (x̂) = 0 ∀j

Convex Case
If f is convex, gi affine, hj convex and Slater’s condition holds (existence of x ∈ C ∩ dom f with hj (x) < 0), then x̂ is a
minimizer iff ∃ν̂, λ̂ such that (x̂, ν̂, λ̂) is a saddle point.

Example
Pn
Let H = Rn , f (x) = i=1 exP , h(x) = 1 − P xi . Constraint set: C = {x ∈ Rn | h(x) ≤ 0}.
i
P
Lagrangian: L(x, λ) = exi + λ(1 − xi ).
• For λ = 0: inf x L = 0
• For λ > 0: unique minimizer is x = (ln(λ), ..., ln(λ)), and

L(λ) = nλ + λ(1 − n ln(λ)) ⇒ λ∗ = exp(1/n), µ = ne1/n

• Unique solution: x∗ = (1/n, ..., 1/n)

Differentiable Case: Fritz-John and KKT


Fritz-John: If f, gi , hj are continuously differentiable and x̂ is a local minimizer, then:
m+q+1
X X
∃(α0 , α1 , ..., αm+q ) ∈ R+ \ {0}, α0 ∇f (x̂) + αi ∇gi (x̂) + αm+j ∇hj (x̂) = 0, αm+j hj (x̂) = 0

KKT Theorem: Assume differentiability and Mangasarian-Fromovitz condition (MFCQ):


• {∇gi (x̂)}m
i=1 is linearly independent

• ∃z such that ⟨∇gi (x̂), z⟩ = 0, ⟨∇hj (x̂), z⟩ < 0 for active j

Then ∃ν̂, λ̂ s.t. x̂ is a critical point of L(·, ν̂, λ̂) and satisfies complementarity.
Remark: MFCQ is satisfied if {∇gi (x̂)} ∪ {∇hj (x̂) | j ∈ J(x̂)} is linearly independent.

You might also like