Lagrangian Duality

José De Doná

September 2004

Centre of Complex Dynamic

Systems and Control
Outline

1 The Lagrangian Dual Problem

Primal and Dual Problems

2 Geometric Interpretation of the Lagrangian Dual

3 Weak Duality

4 Strong Duality
Example

Centre of Complex Dynamic

Systems and Control
Lagrangian Duality

Given a nonlinear programming problem, known as the primal

problem, there exists another nonlinear programming
problem, closely related to it, that receives the name of the
Lagrangian dual problem.

Under certain convexity assumptions and suitable constraint

qualifications, the primal and dual problems have equal
optimal objective values.

Centre of Complex Dynamic

Systems and Control
The Primal Problem

Consider the following nonlinear programming problem:

Primal Problem P

minimise f (x ), (1)
subject to:
gi (x ) ≤ 0 for i = 1, . . . , m,
hi (x ) = 0 for i = 1, . . . ,
,
x ∈ X.

Centre of Complex Dynamic

Systems and Control
The Dual Problem

Then the Lagrangian dual problem is defined as the following

nonlinear programming problem.

Lagrangian Dual Problem D

maximise θ(u, v ), (2)

subject to:
u ≥ 0,

where,

m



θ(u, v ) = inf{f (x ) + ui gi (x ) + vi hi (x ) : x ∈ X }, (3)
i =1 i =1

is the Lagrangian dual function.

Centre of Complex Dynamic

Systems and Control
The Dual Problem

In the dual problem (2)–(3), the vectors u and v have as their

components the Lagrange multipliers ui for i = 1, . . . , m, and
vi for i = 1, . . . ,
.

Note that the Lagrange multipliers ui , corresponding to the

inequality constraints gi (x ) ≤ 0, are restricted to be
nonnegative, whereas the Lagrange multipliers vi ,
corresponding to the equality constraints hi (x ) = 0, are
unrestricted in sign.

Given the primal problem P (1), several Lagrangian dual

problems D of the form of (2)–(3) can be devised, depending
on which constraints are handled as gi (x ) ≤ 0 and hi (x ) = 0,
and which constraints are handled by the set X . (An
appropriate selection of the set X must be made, depending
on the nature of the problem.)
Centre of Complex Dynamic
Systems and Control
Geometric Interpretation
Consider the following primal problem P:

Primal Problem P z
X (g , xxxxxxxxxxxxxxxxxxxxxx
f)
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
minimise f (x ), x xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
subject to: xxxxxxxxxxxxxxxxxxxxxx
G
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
[g (x ), f (x )]
g ( x ) ≤ 0, xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
x ∈ X,

where f : Rn → R and
g : Rn → R. y

Define the following set in R2 :

G = {(y , z ) : y = g (x ), z = f (x ) for some x ∈ X },

that is, G is the image of X under the (g , f ) map.

Centre of Complex Dynamic
Systems and Control
Geometric Interpretation

G = {(y , z ) : y = g (x ), z = f (x ) for some x ∈ X },

z
X (g , xxxxxxxxxxxxxxxxxxxxxx
f)
Primal Problem P xxxxxxxxxxxxxxxxxxxxxx
x xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
G
minimise f (x ), xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
[g (x ), f (x )]
subject to: xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
(ȳ , z̄ ) xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
g ( x ) ≤ 0, xxxxxxxxxxxxxxxxxxxxxx

x ∈ X.
y

Then, the primal problem consists in finding a point in G with y ≤ 0

that has minimum ordinate z .
Obviously this point is (ȳ , z̄ ).
Centre of Complex Dynamic
Systems and Control
Geometric Interpretation
Lagrangian Dual
Problem D z
X (g , xxxxxxxxxxxxxxxxxxxxxx
f)
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
maximise θ(u), x xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
G
subject to: xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
α xxxxxxxxxxxxxxxxxxxxxx
[g (x ), f (x )]
xxxxxxxxxxxxxxxxxxxxxx
u ≥ 0, xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
(ȳ , z̄ )xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx

where (Lagrangian dual z + uy = α

subproblem): Slope −u
y
θ(u) = inf{f (x )+ug (x ) : x ∈ X }.

Given u ≥ 0, the Lagrangian dual subproblem is equivalent to

minimise z + uy over points (y , z ) in G . Note that z + uy = α is the
equation of a straight line with slope −u that intercepts the z -axis
at α.

Centre of Complex Dynamic

Systems and Control
Geometric Interpretation
Lagrangian Dual
z
Problem D X (g , xxxxxxxxxxxxxxxxxxxxxx
f)
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
x xxxxxxxxxxxxxxxxxxxxxx
maximise θ(u), xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
G
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
subject to: xxxxxxxxxxxxxxxxxxxxxx
[g (x ), f (x )]
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
u ≥ 0, xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
(ȳ , z̄ )xxxxxxxxxxxxxxxxxxxxxx

where (Lagrangian dual z + uy = α

θ(u) Slope −u
subproblem):
y

θ(u) = inf{f (x )+ug (x ) : x ∈ X }.

In order to minimise z + uy over G we need to move the line

z + uy = α parallel to itself as far down as possible, whilst it
remains in contact with G . The last intercept on the z -axis thus
obtained is the value of θ(u) corresponding to the given u ≥ 0.

Centre of Complex Dynamic

Systems and Control
Geometric Interpretation
Lagrangian Dual
Problem D z
X (g , xxxxxxxxxxxxxxxxxxxxxx
f)
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
x
maximise θ(u), xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
G
xxxxxxxxxxxxxxxxxxxxxx
subject to: xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
[g (x ), f (x )]
xxxxxxxxxxxxxxxxxxxxxx
u ≥ 0, xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
(ȳ , z̄ ) xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx Slope −ū
where (Lagrangian dual
z + uy = α
subproblem): θ(u) Slope −u
y
θ(u) = inf{f (x )+ug (x ) : x ∈ X }.

Finally, to solve the dual problem, we have to find the line with
slope −u (u ≥ 0) such that the last intercept on the z -axis, θ(u), is
maximal. Such a line has slope −ū and supports the set G at the
point (ȳ , z̄ ). Thus, the solution to the dual problem is ū, and the
optimal dual objective value is z̄ .
Centre of Complex Dynamic
Systems and Control
Geometric Interpretation

z
X (g , xxxxxxxxxxxxxxxxxxxxxx
f)
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
x xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
G
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
[g (x ), f (x )]
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
(ȳ , z̄ ) xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
Slope −ū
z + uy = α
θ(u) Slope −u
y

The solution of the Primal problem is z̄ , and the solution of the

Dual problem is also z̄ .
It can be seen that, in the example illustrated, the optimal
primal and dual objective values are equal. In such cases, it is
said that there is no duality gap (strong duality).
Centre of Complex Dynamic
Systems and Control
Weak Duality

The following result shows that the objective value of any feasible
solution to the dual problem constitutes a lower bound for the
objective value of any feasible solution to the primal problem.

Theorem (Weak Duality Theorem)

Consider the primal problem P given by (1) and its Lagrangian dual
problem D given by (2). Let x be a feasible solution to P; that is,
x ∈ X, g (x ) ≤ 0, and h (x ) = 0. Also, let (u, v ) be a feasible solution
to D; that is, u ≥ 0. Then:

f (x ) ≥ θ(u, v ).
◦

Centre of Complex Dynamic

Systems and Control
Weak Duality

Proof.
We use the definition of θ given in (3), and the facts that x ∈ X ,
u ≥ 0, g (x ) ≤ 0 and h (x ) = 0. We then have

θ(u, v ) = inf{f (x̃ ) + ut g (x̃ ) + v t h (x̃ ) : x̃ ∈ X }

≤ f (x ) + ut g (x ) + v t h (x ) ≤ f (x ),

and the result follows.

Centre of Complex Dynamic

Systems and Control
Weak Duality

We then have, as a corollary of the previous theorem, the following

result.

Corollary

inf{f (x ) : x ∈ X , g (x ) ≤ 0, h (x ) = 0} ≥ sup{θ(u, v ) : u ≥ 0}.

◦

Note from the corollary that the optimal objective value of the
primal problem is greater than or equal to the optimal objective
value of the dual problem.

If the inequality holds as a strict inequality, then it is said that there

exists a duality gap.

Centre of Complex Dynamic

Systems and Control
Weak Duality

The figure shows an example of the geometric interpretation of the

primal and dual problems.
z

X (g , f )xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx Notice that, in the
x xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx case shown in the
G
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx figure, there exists
[g (x ), f (x )]
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx a duality gap due to
Duality gap xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
Optimal primal objective the nonconvexity of
xxxxxxxxxxxxxxxxxxxxxxxxxx
the set G .
Optimal dual objective

We will see, in the Strong Duality Theorem, that if some suitable

convexity conditions are satisfied, then there is no duality gap
between the primal and dual optimisation problems.
Centre of Complex Dynamic
Systems and Control
Strong Duality
Before stating the conditions that guarantee the absence of a
duality gap, we need the following result.
Lemma
Let X be a nonempty convex set in Rn . Let α : Rn → R and
g : Rn → Rm be (componentwise) convex, and h : Rn → R
be
affine (that is, assume h is of the form h (x ) = Ax − b). Also, let u0
be a scalar, u ∈ Rm and v ∈ R
. Consider the following two
systems:
System 1: α(x ) < 0, g (x ) ≤ 0, h (x ) = 0 for some x ∈ X.
System 2: u0 α(x ) + ut g (x ) + v t h (x ) ≥ 0 for some
(u0 , u, v )
(0, 0, 0), (u0 , u) ≥ (0, 0) and for all x ∈ X.
If System 1 has no solution x, then System 2 has a solution
(u0 , u, v ). Conversely, if System 2 has a solution (u0 , u, v ) with
u0 > 0, then System 1 has no solution.

Centre of Complex Dynamic

Systems and Control
Proof of the Lemma

Outline of the proof:

Assume first that

System 1: α(x ) < 0, g (x ) ≤ 0, h (x ) = 0 for some x ∈ X ,
has no solution.
Define the set:

S = {(p , q, r ) : p > α(x ), q ≥ g (x ), r = h (x ) for some x ∈ X }.

The set S is convex, since X , α and g are convex and h is affine.

Since System 1 has no solution, we have that (0, 0, 0)  S .

Centre of Complex Dynamic

Systems and Control
Proof of the Lemma (Ctd.)

Example

h
Consider the functions: 3
{α(x ), h (x ) : x ∈ X }
α(x ) = (x − 1) −
2 1
4,

h (x ) = 2x − 1, α

and the set

X = {x ∈ R : |x | ≤ 2}. −5

Centre of Complex Dynamic

Systems and Control
Proof of the Lemma (Ctd.)

Example (Ctd.)

r
3 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx

α(x ) = (x − 1) −
2 1
4, S

h (x ) = 2x − 1, p

X = {x ∈ R : |x | ≤ 2}.

−5 xxxxx

S = {(p , r ) : p > α(x ), r = h (x ) for some x ∈ X }

Centre of Complex Dynamic

Systems and Control
Proof of the Lemma (Ctd.)

Continuing with the proof of the Lemma, we have the convex set:

S = {(p , q, r ) : p > α(x ), q ≥ g (x ), r = h (x ) for some x ∈ X },

and that (0, 0, 0)  S .

Recall the following corollary of the Supporting Hyperplane

Theorem:

Corollary
Let S be a nonempty convex set in Rn and x̄  int S. Then there is
a nonzero vector p such that p t (x − x̄ ) ≤ 0 for each x ∈ cl S.
◦

Centre of Complex Dynamic

Systems and Control
Proof of the Lemma (Ctd.)

We then have, from the above corollary, that there exists a nonzero
vector (u0 , u, v ) such that

(u0 , u, v )t [(p , q, r ) − (0, 0, 0)] = u0 p + ut q + v t r ≥ 0, (4)

for each (p , q, r ) ∈ cl S .

Now, fix an x ∈ X . Noticing, from the definition of S , that p and q

can be made arbitrarily large, we have that in order to satisfy (4),
we must have u0 ≥ 0 and u ≥ 0.

Centre of Complex Dynamic

Systems and Control
Proof of the Lemma (Ctd.)

Example (Ctd.)

r
3 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx

α(x ) = (x − 1)2 − 14 , (u0 , v ) S

h (x ) = 2x − 1, p

X = {x ∈ R : |x | ≤ 2}.
−5 xxxxxx

We can see that u0 cannot be u0 < 0 and satisfy:

(u0 , v )t [(p , r ) − (0, 0)] = (u0 , v )t (p , r ) = u0 p + v t r ≥ 0,

for each (p , q, r ) ∈ cl S.

Centre of Complex Dynamic

Systems and Control
Proof of the Lemma (Ctd.)
Example (Ctd.)

r
3 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx

α(x ) = (x − 1)2 − 14 , (u0 , v )

h (x ) = 2x − 1, p

S
X = {x ∈ R : |x | ≤ 2}.
−5 xxxxxx

We conclude that u0 ≥ 0 and

(u0 , v )t [(p , r ) − (0, 0)] = (u0 , v )t (p , r ) = u0 p + v t r ≥ 0,

for each (p , q, r ) ∈ cl S.
Centre of Complex Dynamic
Systems and Control
Proof of the Lemma (Ctd.)

We have that there exists a nonzero vector (u0 , u, v ) with

(u0 , u) ≥ (0, 0) such that

(u0 , u, v )t [(p , q, r ) − (0, 0, 0)] = u0 p + ut q + v t r ≥ 0,

for each (p , q, r ) ∈ cl S .

Also, note that [α(x ), g (x ), h (x )] ∈ cl S and we have from the above

inequality that
u0 α(x ) + ut g (x ) + v t h (x ) ≥ 0.
Since the above inequality is true for each x ∈ X , we conclude that
System 2: u0 α(x ) + ut g (x ) + v t h (x ) ≥ 0 for some
(u0 , u, v )
(0, 0, 0), (u0 , u) ≥ (0, 0) and for all x ∈ X .
has a solution.

Centre of Complex Dynamic

Systems and Control
Proof of the Lemma (Ctd.)

To prove the converse, assume that

System 2: u0 α(x ) + ut g (x ) + v t h (x ) ≥ 0 for some

(u0 , u, v )
(0, 0, 0), (u0 , u) ≥ (0, 0) and for all x ∈ X ,

has a solution (u0 , u, v ) such that u0 > 0.

Suppose that x ∈ X is such that g (x ) ≤ 0 and h (x ) = 0.

From the previous inequality we conclude that

u0 α(x ) ≥ −ut g (x ) ≥ 0, since u ≥ 0 and g (x ) ≤ 0. But, since u0 > 0,
we must then have that α(x ) ≥ 0.

Hence,

System 1: α(x ) < 0, g (x ) ≤ 0, h (x ) = 0 for some x ∈ X .

has no solution and this completes the proof.

Centre of Complex Dynamic

Systems and Control
Proof of the Lemma (Ctd.)
Example (Ctd.)

h
{α(x ), h (x ) : x ∈ X }
3

( u0 , v )
α(x ) = (x − 1)2 − 14 ,

h (x ) = 2x − 1, α

X = {x ∈ R : |x | ≤ 2}.
−5


If System 2: u0 α(x ) + v t h (x ) ≥ 0 for some (u0 , v )
(0, 0), u0 ≥ 0

and for all x ∈ X , has a solution such that u0 > 0, and x ∈ X is
such that h (x ) = 0, we can see that α(x ) must be α(x ) ≥ 0, and
hence System 1 has no solution.
Centre of Complex Dynamic
Systems and Control
Strong Duality
The following result, known as the strong duality theorem, shows
that, under suitable convexity assumptions and under a constraint
qualification, there is no duality gap between the primal and dual
optimal objective function values.
Theorem (Strong Duality Theorem)
Let X be a nonempty convex set in Rn . Let f : Rn → R and
g : Rn → Rm be convex, and h : Rn → R
be affine. Suppose that
the following constraint qualification is satisfied. There exists an
x̂ ∈ X such that g (x̂ ) < 0 and h (x̂ ) = 0, and 0 ∈ int h (X ), where
h (X ) = {h (x ) : x ∈ X }. Then,

inf{f (x ) : x ∈ X , g (x ) ≤ 0, h (x ) = 0} = sup{θ(u, v ) : u ≥ 0}, (5)

where θ(u, v ) = inf{f (x ) + ut g (x ) + v t h (x ) : x ∈ X }. Furthermore, if

the inf is finite, then sup{θ(u, v ) : u ≥ 0} is achieved at (ū, v̄ ) with
ū ≥ 0. If the inf is achieved at x̄, then ūt g (x̄ ) = 0.
Centre of Complex Dynamic
Systems and Control
Proof of the Strong Duality Theorem
Let γ = inf{f (x ) : x ∈ X , g (x ) ≤ 0, h (x ) = 0}.
By assumption there exists a feasible solution x̂ for the primal
problem and hence γ < ∞.
If γ = −∞, we then conclude from the corollary of the Weak
Duality Theorem that sup{θ(u, v ) : u ≥ 0} = −∞ and, hence, (5) is
satisfied.
Thus, suppose that γ is finite, and consider the following system:

f (x ) − γ < 0, g (x ) ≤ 0 h ( x ) = 0, for some x ∈ X .

By the definition of γ, this system has no solution. Hence, from the

previous lemma, there exists a nonzero vector (u0 , u, v ) with
(u0 , u) ≥ (0, 0) such that

u0 [f (x ) − γ] + ut g (x ) + v t h (x ) ≥ 0 for all x ∈ X . (6)

Centre of Complex Dynamic

Systems and Control
Proof of the Strong Duality Theorem (Ctd.)

We will next show that u0 > 0. Suppose, by contradiction that

u0 = 0.
By assumption, there exists an x̂ ∈ X such that g (x̂ ) < 0 and
h (x̂ ) = 0. Substituting in (6) we obtain ut g (x̂ ) ≥ 0. But, since
g (x̂ ) < 0 and u ≥ 0, ut g (x̂ ) ≥ 0 is only possible if u = 0.
From (6), u0 = 0 and u = 0 imply that v t h (x ) ≥ 0 for all x ∈ X . But,
since 0 ∈ int h (X ), we can choose an x ∈ X such that h (x ) = −λv ,
where λ > 0. Therefore, 0 ≤ v t h (x ) = −λv 2 , which implies that
v = 0.
Thus, it has been shown that u0 = 0 implies that
(u0 , u, v ) = (0, 0, 0), which is a contradiction. We conclude, then,
that u0 > 0.

Centre of Complex Dynamic

Systems and Control
Proof of the Strong Duality Theorem (Ctd.)

Dividing (6) by u0 and denoting ū = u/u0 and v̄ = v /u0 , we obtain

f (x ) + ūt g (x ) + v̄ t h (x ) ≥ γ for all x ∈ X . (7)

This implies that θ(ū, v̄ ) = inf{f (x ) + ūt g (x ) + v̄ t h (x ) : x ∈ X } ≥ γ.

We then conclude, from the Weak Duality Theorem, that
θ(ū, v̄ ) = γ. And, from the corollary of the Weak Duality Theorem,
we conclude that (ū, v̄ ) solves the dual problem.
Finally, to complete the proof, assume that x̄ is an optimal solution
to the primal problem; that is, x̄ ∈ X , g (x̄ ) ≤ 0, h (x̄ ) = 0 and
f (x̄ ) = γ.
From (7), letting x = x̄ , we get ūt g (x̄ ) ≥ 0. Since ū ≥ 0 and
g (x̄ ) ≤ 0, we get ūt g (x̄ ) = 0.
This completes the proof.

Centre of Complex Dynamic

Systems and Control
Example of Strong Duality

Example
Consider the following optimisation problem:

Primal Problem P

minimise (x − 1)2 ,
subject to:
2x − 1 = 0,
x ∈ X = {x ∈ R : |x | ≤ 2}.

It is clear that the optimal value of the objective function is equal to

 2  
1 1 1
− 1 = , since the feasible set is the singleton .
2 4 2

Centre of Complex Dynamic

Systems and Control
Example of Strong Duality

Example (Ctd.)
Lagrangian Dual Problem D
maximise θ(v ),
where the Lagrangian dual function is,
θ(v ) = inf{(x − 1)2 + v (2x − 1) : |x | ≤ 2}.
Differentiating w.r.t. x and equating to zero, we get that the
optimiser of the dual Lagrangian subproblem is x ∗ = −v + 1 (if
−1 ≤ v ≤ 3).
Hence θ(v ) = (−v + 1 − 1)2 + v (−2v + 2 − 1) = −v 2 + v.
Differentiating w.r.t. v and equating to zero, we get that the
1
optimiser of the dual problem is v ∗ = and the optimal value of
2
∗ ∗ 1
the dual problem is −v + v = . Thus, there is no duality gap.
2
4

Centre of Complex Dynamic

Systems and Control
Example of Strong Duality
Example (Ctd.)

f
X
[ ]
-2 2


[h (x ), f (x )] = 2x − 1, (x − 1)2

-5 3 h

Optimal primal
Slope = −v ∗ = −1/2
and dual objectives

Centre of Complex Dynamic

Systems and Control