ENGG 5501: Foundations of Optimization 2018–19 First Term

Handout 5: Elements of Conic Linear Programming

Instructor: Anthony Man–Cho So October 22, 2018

1 Introduction
As we saw in the previous handout, linear programming is a powerful tool for modelling and
analyzing applications that arise in various disciplines. Despite its wide applicability, however,
there are situations in reality that are inherently nonlinear and cannot be modelled as LPs. In
order to handle such situations, we need to incorporate nonlinear elements in our formulation.
In this lecture, we shall focus on a certain nonlinear extension of LP and demonstrate its power
through various applications.
To motivate our discussion, let us first recall the standard form LP

minimize cT x
subject to Ax = b, (1)
x ≥ 0,

where A ∈ Rm×n , b ∈ Rm , and c ∈ Rn are given. Now, observe that the inequality x ≥ 0 is an
inequality between two vectors. As such, it requires a definition—we say that x ≥ 0 iff xi ≥ 0 for
i = 1, . . . , n. More generally, given two vectors u, v ∈ Rn , we say that u ≥ v iff u − v ≥ 0, or
equivalently, ui ≥ vi for i = 1, . . . , n. It is easy to verify that the relation ≥ defines a partial order
on vectors in Rn ; i.e., it satisfies

(a) (Reflexivity) u ≥ u for all u ∈ Rn ;

(b) (Anti–Symmetry) u ≥ v and v ≥ u imply u = v for all u, v ∈ Rn ;

(c) (Transitivity) u ≥ v and v ≥ w imply u ≥ w for all u, v, w ∈ Rn .

What is also interesting and important in the development of LP duality theory is that the relation
≥ is compatible with linear operations; i.e., it satisfies

(d) (Homogeneity) for any u, v ∈ Rn and α ≥ 0, if u ≥ v, then αu ≥ αv;

(e) (Additivity) for any u, v, w, z ∈ Rn , if u ≥ v and w ≥ z, then u + w ≥ v + z.

At this point one might ask whether ≥ is the only relation that satisfies (a)–(e) above. It turns
out that the answer is no. Indeed, as we shall see, one can introduce nonlinearity into (1) by simply
replacing the constraint x ≥ 0 with the constraint x  0, where  is another relation that satisfies
(a)–(e) above. In order to characterize such relations, let us begin with some notation. Let E be
a finite–dimensional Euclidean space equipped with an inner product • and a relation . We say
that the relation  is good if it satisfies (a)–(e) above. The key observation is that a good relation
 is completely identified by the set K = {u ∈ E : u  0}. In other words, the pairs of vectors
u, v ∈ E for which u  v can be deduced from the set K. Indeed, suppose that u  v. By (a), we

1
have −v  −v, and hence by (e) we have u − v  0. Conversely, if u − v  0, then we can add
v  v to it and obtain u  v.
The upshot of the above observation is that it connects the abstract relation  to a concrete
geometric object K ⊆ E. To take advantage of this connection, let us dig deeper into the structure
of the set K. We claim that K is a pointed cone; i.e., it has the following properties:

1. K is non–empty and closed under addition; i.e., u + v ∈ K whenever u, v ∈ K.

2. K is a cone; i.e., for any u ∈ K and α > 0, we have αu ∈ K.

3. K is pointed ; i.e., if u ∈ K and −u ∈ K, then u = 0.

Indeed, the first property follows from (a) (which implies that 0 ∈ K) and (e); the second follows
from (d). To derive the third property, observe that u  u by (a), which together with −u  0 and
(e) implies that 0  u. Since u  0, it follows from (b) that u = 0.
Note that a pointed cone K is automatically convex. To prove this, let u, v ∈ K and α ∈ (0, 1).
Then, since K is a cone, we have αu, (1 − α)v ∈ K. Since K is closed under addition, we conclude
that αu + (1 − α)v ∈ K as desired.
The above discussion shows that every good relation  on E induces a pointed cone K = {u ∈
E : u  0} with 0 ∈ K. It turns out that the converse is also true; i.e., given an arbitrary pointed
cone K ⊆ E with 0 ∈ K, we can define a good relation on E. To see this, consider the relation K
defined by
u K v ⇐⇒ u − v ∈ K. (2)
Note that by definition, we have u K v iff u − v K 0. Now, we claim that K is good:

(a) (Reflexivity) Since 0 ∈ K, we see that for any u ∈ E, we have u − u ∈ K; i.e., u K u.

(b) (Anti–Symmetry) If u − v ∈ K and v − u ∈ K, then by the pointedness of K, we have

u − v = 0; i.e., u = v.

(c) (Transitivity) If u − v ∈ K and v − w ∈ K, then by the addition property, we have u − w ∈ K;

i.e., u K w.

(d) (Homogeneity) Suppose that u − v ∈ K and α > 0. By the conic property, we have α(u − v) ∈
K, which implies that αu K αv. The case where α = 0 trivially follows from reflexivity.

(e) (Additivity) Suppose that u − v ∈ K and w − z ∈ K. By the addition property, we have

u + w − (v + z) ∈ K; i.e., u + w K v + z.

Thus, the good relations on E are completely characterized by the pointed cones in E that
contain the origin. Let us now consider some particularly interesting examples (in the context of
optimization) of pointed cones and the good relations they induce.

Example 1 (Representative Pointed Cones)

1. Non–Negative Orthant. Rn+ = {x = (x1 , . . . , xn ) ∈ Rn : xi ≥ 0 for i = 1, . . . , n}.PWe con-

sider Rn+ as a pointed cone in Rn equipped with the usual inner product; i.e., uT v = ni=1 ui vi .
The good relation it induces is the usual coordinate–wise ordering: For u, v ∈ Rn , we have
u ≥ v iff ui ≥ vi for i = 1, . . . , n.

2
 2. Lorentz Cone (also known as the Second–Order Cone or the Ice Cream Cone).
Qn+1 = {(t, x) ∈ R × Rn : t ≥ kxk2 }. We consider Qn+1 as a pointed cone in Rn+1 equipped
with the usual inner product. The good relation it induces is the following: For (s, u), (t, v) ∈
R × Rn , we have (s, u) Qn+1 (t, v) iff s − t ≥ ku − vk2 .
n = X ∈ S n : uT Xu ≥ 0 for all u ∈ Rn . We consider S n

3. Positive Semidefinite Cone. S+ +
as a pointed cone in the space S n of n × n symmetric matrices equipped with the Frobenius
inner product; i.e.,
n X
X n
T
X • Y = tr(X Y ) = tr(XY ) = Xij Yij
i=1 j=1

(note that S n can be identified with Rn(n+1)/2 ). The good relation it induces is the so–called
positive semidefinite ordering: For X, Y ∈ S n , we have X  Y iff X − Y is positive semide-
finite (denoted by X − Y  0).
Note that all the cones in Example 1 are closed and have non–empty interiors. These properties
have important consequences. First, the closedness of K implies that we can take limits in the
relation K . Specifically, if {ui }, {v i } are sequences in E such that
ui K v i for i = 1, 2, . . . ; ui → u ∈ E; v i → v ∈ E,
then we may conclude that u K v. Second, if the pointed cone K has a non–empty interior, then
in addition to the non–strict relation K , we may define a strict relation K via u K v ⇐⇒
u − v ∈ int(K). As we shall see, such a relation is useful in the development of the duality theory
for conic optimization problems.
Before we leave this section, let us introduce a useful operation, which allows us to create new
pointed cones from old ones.
Proposition 1 Let E1 , . . . , En be finite–dimensional Euclidean spaces and Ki ⊆ Ei be closed poin-
ted cones with non–empty interiors, where i = 1, . . . , n. Then, the set
K ≡ K1 × · · · × Kn = {(x1 , . . . , xn ) ∈ E1 × · · · × En : xi ∈ Ki for i = 1, . . . , n}
is a closed pointed cone with non–empty interior.
We leave the proof as a simple exercise to the reader.

2 Conic Linear Programming

As before, let E be a finite–dimensional Euclidean space equipped with an inner product • and a
good relation . Motivated by the development in the previous section, we can define an analog
of the LP (1) by defining linear functions using • and replacing the constraint x ≥ 0 with x  0.
More precisely, let K ⊆ E be the pointed cone induced by  and suppose that K is closed with
non–empty interior. Given c, a1 , . . . , am ∈ E and b1 , . . . , bm ∈ R, we define the standard form
Conic Linear Programming (CLP) problem as follows:
vp∗ = inf c•x
subject to ai • x = bi for i = 1, . . . , m, (P )
x K 0.

3
The term “conic linear programming” comes from the fact that Problem (P ) involves minimizing
a linear objective function subject to linear equality constraints and a conic constraint. Note that
nonlinearity can be incorporated into (P ) through the pointed cone K. As we shall see, this gives
Problem (P ) tremendous modeling power.
To develop the dual of Problem (P ), it is natural to follow the strategy used in the LP case. In
the current setting, our goal is to find a vector y ∈ Rm such that
m m
!
X X
bT y = (ai • x)yi = yi ai • x ≤ c • x. (3)
i=1 i=1

Pmcase of LP, we have x ≥ 0. Thus, we can ensure that the above inequality holds by requiring
For the
c − i=1 yi ai ≥ 0. For the general case of CLP, however, we only have x K 0. What then is
the P
condition that can ensure the validity of inequality (3)? One obvious answer is that the vector
c− m i=1 yi ai should belong to the set

K ∗ = {w ∈ E : x • w ≥ 0 for all x ∈ K} .
The set K ∗ is called the dual cone of the cone K and turns out to have a number of important
properties. We summarize these properties in the following proposition:
Proposition 2 Let K ⊆ E be a non–empty set. Then, the following hold:
(a) The set K ∗ is a closed convex cone.
(b) If K is a closed convex cone, then so is K ∗ . Moreover, we have (K ∗ )∗ = K.
(c) If K has a non–empty interior, then K ∗ is pointed.
(d) If K is a closed pointed cone, then K ∗ has a non–empty interior.
Proof The proof of (a) is straightforward and is left as an exercise to the reader.
Next, we prove (b). It is clear from the definition that K ⊆ (K ∗ )∗ . To establish the converse,
let v ∈ (K ∗ )∗ be arbitrary. If v 6∈ K, then by the separation theorem (Theorem 7 of Handout 2),
there exists a y ∈ Rn such that inf x∈K y T x > y T v. We claim that θ∗ ≡ inf x∈K y T x = 0. Clearly, we
have θ∗ ≤ 0, since 0 ∈ K. Now, if θ∗ < 0, then there exists an x0 ∈ K such that 0 > y T x0 > y T v.
However, since αx0 ∈ K for all α > 0, we see that αy T x0 > y T v for all α ≥ 1, which is impossible.
Thus, the claim is established. In particular, this shows that y ∈ K ∗ . However, we then have the
inequality 0 > y T v, which contradicts the fact that v ∈ (K ∗ )∗ . Hence, we conclude that v ∈ K.
To prove (c), suppose that K ∗ is not pointed. Then, there exists a w ∈ K ∗ such that w 6= 0
and x • w = 0 for all x ∈ K. This implies that K is a subset of the hyperplane H(w, 0) = {x ∈ E :
w • x = 0}, which shows that int(K) = ∅.
Lastly, let us prove (d). Suppose that int(K ∗ ) = ∅. Then, there exists a hyperplane H(s, 0) =
{w ∈ E : s • w = 0} with s 6= 0 such that K ∗ ⊆ H(s, 0). Since K is a closed convex cone by
assumption, using the result in (b), we compute
K = (K ∗ )∗ = {x ∈ E : x • w ≥ 0 for all w ∈ K ∗ }
⊃ {x ∈ E : x • w ≥ 0 for all w ∈ H(s, 0)}
= {λs : λ ∈ R}.
This shows that K is not pointed. u
t
In view of Proposition 2, the following is immediate:

4
Corollary 1 Let K ⊆ E be a closed pointed cone with non–empty interior. Then, the dual cone
K ∗ ⊆ E is also a closed pointed cone with non–empty interior.

We also have the following proposition, which shows that the dual of a Cartesian product of
cones is simply the Cartesian product of the corresponding dual cones:

Proposition 3 Let E1 , . . . , En be finite–dimensional Euclidean spaces equipped with the inner pro-
ducts •1 , . . . , •n , respectively. Let E = E1 × · · · × En and define the inner product • on E by
n
X
u•v = ui •i vi where ui , vi ∈ Ei ; i = 1, . . . , n.
i=1

Suppose that Ki ⊆ Ei (where i = 1, . . . , n) are closed pointed cones with non–empty interiors and
let K = K1 × · · · × Kn . Then, the dual cone K ∗ is given by K ∗ = K1∗ × · · · × Kn∗ and is a closed
pointed cone with non–empty interior.

Proof Let (y1 , . . . , yn ) ∈ K ∗ . Then, we have ni=1 xi •i yi ≥ 0 for all (x1 , . . . , xn ) ∈ K. We claim
P
that xi •i yi ≥ 0 for all xi ∈ Ki , where i = 1, . . . , n. Indeed, suppose that this is not the case. Then,
there exists a j ∈ {1, . . . , n} and an x̄j ∈ Kj such that x̄j •j yj < 0. Since Kj is a cone, we have
αx̄j ∈ K for any α > 0. Now, let x0i ∈ Ki be arbitrary, where i ∈ {1, . . . , j − 1, j + 1, . . . , n}. Then,
we have X
x0i •i yi + (αx̄j ) •j yj < 0
i6=j

for sufficiently large α, which is a contradiction. Hence, we have xi •i yi for all xi ∈ Ki , where
i = 1, . . . , n. In particular, we have (y1 , . . . , yn ) ∈ K1∗ × · · · × Kn∗ .
Conversely, suppose that (y1 , . .P . , yn ) ∈ K1∗ × · · · × Kn∗ . Then, we have xi •i yi ≥ 0 for all xi ∈ Ki ,
where i = 1, . . . , n. It follows that ni=1 xi •i yi ≥ 0 for all (x1 , . . . , xn ) ∈ K; i.e., (y1 , . . . , yn ) ∈ K ∗ .
The claim that K ∗ is a closed pointed cone with non–empty interior follows from the assumption
on K and Corollary 1. u
t
The above discussion leads us to the following dual of Problem (P ), whose objective value gives
the largest lower bound on vp∗ :

vd∗ = sup bT y
m
X
subject to yi ai + s = c, (D)
i=1

y ∈ Rm , s K ∗ 0.

By Corollary 1, if K is a closed pointed cone with non–empty interior, then so is K ∗ . In this case,
both (P ) and (D) are of the same nature; i.e., they both involve optimizing a linear function over
a set defined by linear equality constraints and a conic constraint that is associated with a closed
pointed cone with non–empty interior.
Before we study the relationship between the primal–dual pair of problems (P ) and (D), let us
consider some concrete instances of them.

Example 2 (Representative CLP Problems)

5
1. Linear Programming (LP). By taking E = Rn , K = Rn+ , and u • v = uT v for u, v ∈ E,
Problem (P ) becomes

inf cT x
subject to aTi x = bi for i = 1, . . . , m,
x ∈ Rn+ ,

which is nothing but an LP in standard primal form. To obtain an explicit description of its
dual problem (D), we need to compute K ∗ = (Rn+ )∗ . We claim that Rn+ is self–dual; i.e.,
(Rn+ )∗ = Rn+ . Indeed, on one hand, we have Rn+ ⊆ (Rn+ )∗ because y T x ≥ 0 if x, y ≥ 0. On the
other hand, suppose that y ∈ (Rn+ )∗ . Then, we have xT y ≥ 0 for all x ∈ Rn+ . In particular,
we have eTi y = yi ≥ 0 for i = 1, . . . , n, where ei ∈ Rn is the i–th standard basis vector. This
shows that y ∈ Rn+ , as desired.
Based on the above discussion, we see that Problem (D) becomes

sup bT y
Xm
subject to yi ai + s = c,
i=1
y ∈ Rm , s ∈ Rn+ ,

which is an LP in standard dual form.

2. Second–Order Cone Programming (SOCP). Let E = Rn+1 , K = Qn+1 , and u•v = uT v

for u, v ∈ E. Then, Problem (P ) becomes

inf cT x
subject to aTi x = bi for i = 1, . . . , m,
x∈ Qn+1 ,

which is an SOCP in standard primal form. To derive its dual, we need to compute (Qn+1 )∗ . It
is an easy exercise to show that Qn+1 is self–dual; i.e., (Qn+1 )∗ = Qn+1 . Thus, Problem (D)
becomes
sup bT y
Xm
subject to yi ai + s = c, (4)
i=1
y ∈ Rm , s ∈ Qn+1 ,
which is an SOCP in standard dual form. To get a better understanding of Problem (4), let
us write it out explicitly. Towards that end, let ai = (ui , ai,1 , . . . , ai,n ) ∈ Rn+1 for i = 1, . . . , m
and c = (v, d) ∈ Rn+1 with d ∈ Rn . Then, we have
m
X
yi ai = (uT y, AT y),
i=1

where A ∈ Rm×n is the matrix

Pmwhose i–th n+1
row contains the entries ai,1 , . . . , ai,n . It follows
that the constraint s = c − i=1 yi ai ∈ Q is equivalent to (v − uT y, d − AT y) ∈ Qn+1 ,

6
 which implies that Problem (4) takes the form

sup bT y
(5)
subject to (v − uT y, d − AT y) ∈ Qn+1 .

In other words, the problem of optimizing a linear function subject to the constraint that the
image of an affine map belongs to a second–order cone is an SOCP.
A natural extension of Problem (5) is to allow multiple second–order cone constraints; i.e.,

sup bT y
(6)
subject to (vj − (uj )T y, dj − (Aj )T y) ∈ Qnj +1 for j = 1, . . . , p,

where uj ∈ Rm , vj ∈ R, Aj ∈ Rm×nj , and dj ∈ Rnj for i = 1, . . . , p. This can also be viewed

as an SOCP, as it can be put into the form

sup bT y
Xm
subject to yi āi + s̄ = c̄,
i=1
y ∈ Rm , s ∈ Qn1 +1 × · · · × Qnp +1 .

Here, c̄ = (c1 , . . . , cp ), s̄ = (s1 , . . . , sp ), and āi = (a1i , . . . , api ) are vectors in Rn1 +1 ×· · ·×Rnp +1 ,
with
cj = (vj , dj ) ∈ Rnj +1 and aji = (uji , aji,1 , . . . , aji,nj ) ∈ Rnj +1 .

Using (6), it is immediate that the class of SOCPs includes the class of LPs as a special case.
Indeed, the standard form LP (1) can be formulated as the following SOCP:

inf cT x
subject to kAx − bk2 ≤ 0,
k0x − 0k2 ≤ eTi x for i = 1, . . . , n.

3. Semidefinite Programming (SDP). By taking E = S n , K = S+

n , X • Y = tr(X T Y ) for

X, Y ∈ E, Problem (P ) becomes

inf C •X
subject to Ai • X = bi for i = 1, . . . , m,
X∈ n,
S+

which is an SDP in standard primal form. We leave it as an exercise to the reader to show
n is also self–dual; i.e., (S n )∗ = S n . Based on this result, Problem (D) becomes
that S+ + +

sup bT y
Xm
subject to yi Ai + S = C,
i=1
y ∈ Rm , S ∈ S+
n,

7
 which is an SDP in standard dual form.
Note that the class of SDPs includes the class of SOCPs as a special case. To see this, we
first observe that the SOCP (5) is equivalent to
sup bT y
" #
(v − uT y)I d − AT y (7)
subject to  0.
(d − AT y)T v− uT y

Indeed, let ȳ ∈ Rm be a feasible solution to Problem (5). Then, we have v − uT ȳ ≥ 0. If

v − uT ȳ = 0, then d − AT ȳ = 0, which shows that ȳ ∈ Rm is feasible for (7). On the other
hand, if v − uT ȳ > 0, then we can write the constraint in (5) as
kd − AT ȳk22
v − uT ȳ − ≥ 0, (8)
v − uT ȳ
which, by the Schur complement, is equivalent to
" #
(v − uT ȳ)I d − AT ȳ
 0; (9)
(d − AT ȳ)T v − uT ȳ
i.e., ȳ ∈ Rm is feasible for (7).
Conversely, let ȳ ∈ Rm be a feasible solution to Problem (7). Then, we have (v − uT ȳ)I  0,
which implies that v − uT ȳ ≥ 0. Now, if v − uT ȳ = 0, then (v − uT ȳ)I = 0, which, by (9),
implies that d − AT ȳ = 0. On the other hand, if v − uT ȳ > 0, then constraint (9) is equivalent
to constraint (8). In either case, we see that ȳ ∈ Rm is feasible for (4). Hence, we conclude
that (5) and (7) are equivalent.
We now leave it as an exercise to the reader to show that Problem (7) can be expressed as an
SDP in standard dual form.
Let us now return to study the relationship between the primal–dual pair of CLPs (P ) and (D).
Our first result is the CLP Weak Duality Theorem, which essentially follows from our construction
of (D).
Theorem 1 (CLP Weak Duality) Let x̄ ∈ K be feasible for (P ) and (ȳ, s̄) ∈ Rm ×K ∗ be feasible
for (D). Then, bT ȳ ≤ c • x̄.
Proof We compute
m m
!
X X
c • x̄ = ȳi ai + s̄ •x= ȳi (ai • x) + s̄ • x̄ = bT ȳ + s̄ • x̄ ≥ bT ȳ,
i=1 i=1

where the last inequality follows from the fact that x̄ ∈ K and s̄ ∈ K ∗ . This completes the proof.
u
t
To establish strong duality between (P ) and (D), one natural approach is to develop a conic
version of the Farkas lemma. Towards that end, recall that the classic Farkas lemma is concerned
with the following alternative linear systems:
(A) aTi x = bi for i = 1, . . . , m, x ∈ Rn+ .
m
X
(B) − yi ai ∈ Rn+ , bT y > 0,
i=1

8
where a1 , . . . , am ∈ Rn and b ∈ Rm are given. This suggests that a Farkas–type lemma should hold
for the following conic linear systems:

(I) ai • x = bi for i = 1, . . . , m, x ∈ K.
m
X
(II) − yi ai ∈ K ∗ , bT y > 0.
i=1

Here, as before, E is a finite–dimensional Euclidean space equipped with an inner product •, K ⊆ E

is a closed pointed cone with non–empty interior, and a1 , . . . , am ∈ E and b ∈ Rm are given vectors.
It is straightforward to show that (I) and (II) cannot both have solutions. Indeed, suppose that
x̄ ∈ E is a solution to (I) and ȳ ∈ Rm is a solution to (II). Then, we have
m m
" ! #
X X
T
0<b y= yi (ai • x) = − − yi ai • x ≤ 0,
i=1 i=1

which is a contradiction. Unfortunately, as the following example shows, it is possible that neither
(I) nor (II) has a solution.

Example 3 (“Failure” of the Conic Farkas Lemma) Let E = S 2 and K = S+

2 . Define

" # " # " #

1 0 0 1 0
A1 = , A2 = , b= .
0 0 1 0 2

Consider the following systems:

" #
X11 X12 2
(I) A1 • X = b1 , A2 • X = b2 , X = ∈ S+ .
X12 X22
2
(II) −(y1 A1 + y2 A2 ) ∈ S+ , bT y > 0.

Observe that (I) is equivalent to

2
X11 = 0, X12 = 1, X ∈ S+

and (II) is equivalent to " #

y1 y2 2
− ∈ S+ , y2 > 0,
y2 0
both of which are insolvable.

To see why a Farkas–type lemma need not hold for the systems (I) and (II), recall that in the
proof of Farkas’ lemma for linear systems, we show that the set S = {Ax ∈ Rm : x ∈ Rn+ } is closed
and convex, so that we can apply the separation theorem to {b} and S if system (A) is insolvable.
However, for a general closed pointed cone K, the set S 0 = {(a1 • x, . . . , am • x) ∈ Rm : x ∈ K}
need not be closed. Indeed, in the setting of Example 3, we have

S 0 = (A1 • X, A2 • X) : X ∈ S+
2 2
 
= (X11 , 2X12 ) : X ∈ S+ .

9
 2 if and only if all of its principal minors are non–negative, it can be verified that
Since X ∈ S+

S 0 = {(0, 0)} ∪ {(x, y) : x > 0, y ∈ R} ⊂ R2 ,

which is clearly not closed. Such observation raises a natural question: When is the linear image
of a closed convex cone closed? In [2], Pataki addressed this question and gave a fairly complete
answer. We shall not discuss Pataki’s results here and refer the interested reader to his paper.
The above discussion shows that the systems (I) and (II) may not be alternatives to each other.
However, not all is lost. In fact, by imposing extra conditions, one can establish a conic version of
the Farkas lemma for systems (I) and (II).

Theorem 2 (Conic Farkas’ Lemma) Let E be a finite–dimensional Euclidean space equipped

with an inner product •, K ⊆ E be a closed pointed cone with non–empty interior, and a1 , . . . , am ∈
E and b ∈ RPm be given vectors. Suppose that the Slater condition holds; i.e., there exists a ȳ ∈ Rm

such that − m ∗
i=1 ȳi ai ∈ int(K ). Then, exactly one of the systems (I) and (II) has a solution.

Using Theorem 2, we can establish the following CLP Strong Duality Theorem:

Theorem 3 (CLP Strong Duality)

(a) Suppose that (P ) is bounded below and strictly feasible; i.e., there exists a feasible solution
x̄ to (P ) such that x̄ ∈ int(K). Then, we have vp∗ = vd∗ . Moreover, there exists a feasible
solution (ȳ, s̄) to (D) such that bT ȳ = vd∗ = vp∗ ; i.e., the common optimal value is attained by
some dual feasible solution.

(b) Suppose that (D) is bounded above and strictly feasible; i.e., there exists a feasible solution
(ȳ, s̄) to (D) such that s̄ ∈ int(K ∗ ). Then, we have vp∗ = vd∗ . Moreover, there exists a feasible
solution x̄ to (P ) such that c • x̄ = vp∗ = vd∗ ; i.e., the common optimal value is attained by
some primal feasible solution.

(c) Suppose that either (P ) or (D) is bounded and strictly feasible. Then, given a feasible solution
x̄ to (P ) and a feasible solution (ȳ, s̄) to (D), the following are equivalent:

• x̄ and (ȳ, s̄) are optimal for (P ) and (D), respectively.

• The duality gap is zero; i.e., c • x̄ = bT ȳ.
• We have complementary slackness; i.e., x̄ • s̄ = 0.

Upon closer inspection, we see that the CLP Strong Duality Theorem is weaker than the LP
Strong Duality Theorem. Indeed, recall from Handout 3 that in the case of LP, whenever one
of (P ) or (D) is bounded and feasible, then (i) the optimal value vp∗ of (P ) and the optimal value
vd∗ of (D) are equal, and (ii) there exists a primal feasible solution x̄ and a dual feasible solution
(ȳ, s̄) such that cT x̄ = vp∗ = vd∗ = bT ȳ. In other words, primal and dual attainment of the common
optimal value is implied by the boundedness and feasibility of either the primal or the dual LP
problem. However, such a claim is absent from the statement of Theorem 3. It is thus natural to
ask whether one can strengthen the conclusion of Theorem 3 so that it directly generalizes the LP
Strong Duality Theorem. Curiously, as the following examples show, several things can go wrong
in the case of CLP:

Example 4 (Pathologies in Conic Duality)

10
 1. Both the primal problem (P ) and the dual problem (D) are bounded and feasible, but the
duality gap is non–zero.

Consider the SDP

inf X12
 
0 X12 0 (10)
subject to X =  X12 X22 0 3.
 ∈ S+
0 0 1 + X12

It is a routine exercise to show that the dual of (10) is given by

sup y4
 
−y1 (1 + y4 )/2 −y2 /2 (11)
subject to S =  (1 + y4 )/2 0 3.
−y3 /2  ∈ S+
−y2 /2 −y3 /2 −y4

Since X ∈ S+ 3 , we must have X

12 = 0, which implies that the optimal value of (10) is 0.
Similarly, since S ∈ S+3 , we must have (1 + y )/2 = 0, or equivalently, y = −1. Hence, the
4 4
optimal value of (11) is −1.

2. The primal problem (P ) is bounded below and strictly feasible, but the optimal value is not
attained by any primal feasible solution.

Consider the SOCP

inf x1
(12)
subject to (x1 + x2 , 1, x1 − x2 ) ∈ Q3 .
Note that the constraint in (12) is equivalent to
p
x1 + x2 ≥ 1 + (x1 − x2 )2 ,

which in turn is equivalent to

4x1 x2 ≥ 1, x1 + x2 > 0. (13)
By (13), we see that the optimal value of (12) is bounded below by 0. Moreover, for x1 =
x2 = 1, we have (2, 1, 0) ∈ int(Q3 ), which implies that (12) is strictly feasible. Now, by setting
x1 = 1/(4x2 ) and letting x2 → ∞, we see that the optimal value of (12) is 0. However, such
an optimal value is not attained by any feasible solution to (12).
It is instructive to inspect the dual of (12), which is given by

sup −y2
subject to y1 + y3 = 1,
(14)
y1 − y3 = 0,
(y1 , y2 , y3 ) ∈ Q3 .

The feasible set of (14) is {(1/2, 0, 1/2)}, which shows that the optimal value of (14) is 0.
However, it is clear that (14) is not strictly feasible.
For further results on CLP duality, we refer the reader to [1, 3, 4].

11
References
[1] Z.-Q. Luo, J. F. Sturm, and S. Zhang. Duality and Self–Duality for Conic Convex Programming.
Technical Report No. EI 9620/A, Econometric Institute, Erasmus University Rotterdam, The
Netherlands, 1996.

[2] G. Pataki. On the Closedness of the Linear Image of a Closed Convex Cone. Mathematics of
Operations Research, 32(2):395–412, 2007.

[3] I. Pólik and T. Terlaky. Exact Duality for Optimization over Symmetric Cones. Technical
Report No. 2007/10, Advanced Optimization Laboratory, McMaster University, Hamilton, ON,
Canada, 2007.

[4] S. P. Schurr, A. L. Tits, and D. P. O’Leary. Universal Duality in Conic Convex Optimization.
Mathematical Programming, Series A, 109(1):69–88, 2007.

12