Linear Programming Duality

P&S Chapter 3
Last Revised – Nov 1, 2004

1
In this section we lean about duality, which is another
way to approach linear programming. In particular, we
will see:

• How to define the dual of a normal (primal) linear

program.
If the primal is a minimization problem, the dual is
a maximization problem.

• That the optimum solution to the dual has cost

equal to that of the optimum solution to the primal.

• Complementary Slackness:
A combinatorial statement of the relationship be-
tween the primal and the dual

• A primal-dual interpretation of the shortest path

problem.

2
 min c′x
Convert an LP in general
form such as one on the a′ix = bi i ∈ M
right into a LP in stan- a′ix ≥ bi i ∈ M̄
dard form using method xj ≥ 0 j ∈ N
seen earlier. That is xj ≷ 0 j ∈ N̄

min ĉ′x̂
Âx̂ = b
x̂ ≥ 0
where
" #
0, i ∈ M
 = Aj , j ∈ N (Aj , −Aj ), j ∈ N̄
−I, i ∈ M̄

x̂ = col(xj , j ∈ N |(x+
j , x − s
j ), j ∈ N̄ |xi , i ∈ M̄ )

ĉ = col(cj , j ∈ N |(cj , −cj ), j ∈ N̄ |0)

3
  
0, i ∈ M
min ĉ′x̂ Â = Aj , j ∈ N (Aj , −Aj ), j ∈ N̄
−I, i ∈ M̄
Âx̂ = b x̂ = col(xj , j ∈ N |(x+ − s
j , xj ), j ∈ N̄ |xi , i ∈ M̄ )
x̂ ≥ 0
ĉ = col(cj , j ∈ N |(cj , −cj ), j ∈ N̄ |0)

Recall that BFS x̂0 is optimal iff corresponding c̄ ≥ 0.

If this occurs, there is a corresponding basis B̂ s.t.
′ ′ −1
 
ĉ − ĉB B̂ Â ≥ 0

so π ′ = ĉ′B B̂ −1 is a feasible solution to

π ′Â ≤ ĉ′
where π ∈ Rm and m = |M | + |M̄ | is # of rows in
original LP.

Note that there are three different sets of inequalities,

one each for j ∈ N, j ∈ N̄ , and i ∈ M̄ .

4
  
0, i ∈ M
min ĉ′x̂ Â = Aj , j ∈ N (Aj , −Aj ), j ∈ N̄
−I, i ∈ M̄
Âx̂ = b x̂ = col(xj , j ∈ N |(x+ − s
j , xj ), j ∈ N̄ |xi , i ∈ M̄ )
x̂ ≥ 0
ĉ = col(cj , j ∈ N |(cj , −cj ), j ∈ N̄ |0)

which we saw leads to π ′Â ≤ ĉ′

1. If j ∈ N then π ′Aj ≤ cj

2. If j ∈ N̄ then π ′Aj ≤ cj and −π ′Aj ≤ −cj so

π ′Aj = cj.

3. If i ∈ M̄ then −π ′ ≤ 0 so π ′ ≥ 0.

Using these equations, given a primal LP in general

form, we can define a new LP in dual form.

Primal Dual
min c′x max π ′b
a′ix = bi i∈M πi ≷ 0
a′ix ≥ bi i ∈ M̄ πi ≥ 0
xj ≥ 0 j∈N π ′Aj ≤ cj
xj ≷ 0 j ∈ N̄ π ′Aj = cj

5
 Primal Dual
min c′x max π ′b
a′ix = bi i∈M πi ≷ 0
a′ix ≥ bi i ∈ M̄ πi ≥ 0
xj ≥ 0 j∈N π ′Aj ≤ cj
xj ≷ 0 j ∈ N̄ π ′Aj = cj

Theorem: If an LP has an optimal solution, so does

its dual and, at optimality, their costs are equal.

Proof: Let x, π be feasible solutions to the primal and

dual. Then
c′x ≥ π ′Ax ≥ π ′b (1)
Since primal has optimal solution, dual can not have
unbounded feasible solutions. We saw before that, if
x̂0 is optimal in primal then π ′ = ĉ′B B̂ −1 is, by con-
struction, feasible in dual. This means dual is bounded
and has some feasible solution so, by simplex algo-
rithm, dual has an optimal (bounded) solution (BFS).

The cost of this particular π ′ is

π ′b = ĉ′B B̂ −1b = ĉ′B x̂0
Therefore, by (1), π ′ is optimal in dual.
6
 Primal Dual
min c′ x max π ′ b
a′ix = bi i∈M πi ≷ 0
a′ix ≥ bi i ∈ M̄ πi ≥ 0
xj ≥ 0 j∈N ′
π Aj ≤ cj
xj ≷ 0 j ∈ N̄ π ′ Aj = cj

Theorem: The dual of the dual is the primal.

Proof: Write dual as

min π ′(−b)
(−A′j )π ≥ −cj j∈N
(−Aj )π = −cj j ∈ N̄
πi ≥ 0 i ∈ M̄
πi ≷ 0 i∈M
and consider it as primal. Then

max x′(−c)
xj ≥ 0 j∈N
xj ≷ 0 j ∈ N̄
−a′ix ≤ −b i ∈ M̄
−a′ix = −b i∈M

7
Theorem: Given a primal-dual pair, exactly one of the
three situations occurring below occurs. The X’s de-
note situations which can not occur.

Dual Finite Unbounded Infeasible

Primal optimum

Finite 1 X X
optimum

Unbounded X X 3

Infeasible X 3 2

Proof:We already saw that (1) is possible. We also

saw that if primal has finite optimum, then the dual
must have a finite optimum so the X’s in the first row
are also correct. Since the dual of the dual is the pri-
mal we find, by symmetry, that the X’s in the first col-
umn are correct.

We also saw that any feasible solution to the problem

upper bounds the cost of all feasible solutions to the
dual so it is impossible for both of them to simultane-
ously have unbounded solutions. We will see exam-
ples of (2) and (3) on the next slide.
8
 PRIMAL DUAL
min x1 max π1 + π2
x1 + x2 ≥ 1 π1 − π2 = 1
−x1 − x2 ≥ 1 π1 − π2 = 0
x1 ≷ 0 π1 ≥ 0
x2 ≷ 0 π2 ≥ 0

Note that both the Primal and Dual are infeasible, giv-
ing case 2. Now modify the primal so that x1, x2 ≥ 0.
Then

PRIMAL DUAL
min x1 max π1 + π2
x1 + x2 ≥ 1 π1 − π2 ≤ 1
−x1 − x2 ≥ 1 π1 − π2 ≤ 0
x1 ≥ 0 π1 ≥ 0
x2 ≥ 0 π2 ≥ 0

and the primal is infeasible while the dual is unbounded

(case 3). Primal unbounded and dual infeasible fol-
lows by flipping the primal and the dual.
9
Recall the Diet Problem:
min c′x
Ax ≥ r
x ≥ 0

ai,j = amount of ith nutrient in a unit of the jth food

i = 1, . . . , m, j = 1, . . . , n
ri = yearly requirement of ith nutrient
i = 1, . . . , m
xj = yearly consumption of the jth food (in units)
j = 1, . . . , n
cj = cost per unit of the jth food
j = 1, . . . , n

The Dual is
max π ′r
π ′A ≤ c′
π′ ≥ 0
This can be interpreted as saying that a pill-manufacturer
wants to sell m nutrients; pill i containing one unit of
nutrient i at price πi. Manufacturer wants to maximize
revenue. Constraint is that, for each food j, it must
be cheaper for customer to satisfy nutritional require-
ments via pills rather than eating food.
10
 Complementary Slackness Conditions

Theorem: A pair x, π respectively feasible in a primal-

dual pair are optimal iff

ui = πi(a′i x − bi) = 0 for all i (2)

vj = (cj − π ′Aj )xj = 0 for all j (3)

Proof: By duality definitions

∀i, ui ≥ 0 and ∀j, vj ≥ 0. Now set
X X
u= ui ≥ 0 and v = vj ≥ 0
i j
Then u = 0 iff (2) and v = 0 iff (3).

Now note that

u + v = c′x − π ′b.
So, (2) and (3) are true iff c′x = π ′b which is true iff
both x and π are optimal.
11
 Example

min x1 + x2 + x3 + x4 + x5
3x1 + 2x2 + x3 = 1
Primal 5x1 + x2 + x3 + x4 = 3
2x1 + 5x2 + x3 + x5 = 4
xi ≥ 0

max π1 + 3π2 + 4π3

3π1 + 5π2 + 2π3 ≤ 1
2π1 + π2 + 5π3 ≤ 1
Dual π1 + π2 + π3 ≤ 1
π2 ≤1
π3 ≤ 1
πi ≷ 0 for all i

Because primal is in standard form we have

∀i, ui = πi(a′i x − bi) = 0.
We now need to satisfy
∀j, vj = (cj − π ′Aj )xj = 0.

12
 min x1 + x2 + x3 + x4 + x5
3x1 + 2x2 + x3 = 1
Primal 5x1 + x2 + x3 + x4 = 3
2x1 + 5x2 + x3 + x5 = 4
xi ≥ 0

max π1 + 3π2 + 4π3

3π1 + 5π2 + 2π3 ≤ 1
2π1 + π2 + 5π3 ≤ 1
Dual π1 + π2 + π3 ≤ 1
π2 ≤1
π3 ≤ 1
πi ≷ 0 for all i
We now need to satisfy
∀j, vj = (cj − π ′Aj )xj = 0.
Optimal primal is (0, 1/2, 0, 5/2, 3/2) (with cost 9/2)
so need equality in 2nd, 4th and 5th equations, i.e.,
c2 − π ′A2 = 0 or 2π1 + π2 + 5π3 = 1
c4 − π ′A4 = 0 or π2 = 1
c5 − π ′A5 = 0 or π3 = 1
which has solution
(π1, π2, π3) = (−5/2, 1, 1).
Note that this has same cost 9/2 as primal and is
therefore optimal.
13
 The Shortest Path problem & its dual

Let G = (V, E) be a directed graph s.t. every edge

ej ∈ E has cost cj ≥ 0. The shortest path problem
is to find a directed path from source s to sink t with
minimal cost. We will now see how to write this as an
LP and then derive its dual.

The feasible set of this problem is

F = { sequences P = (ej1 , . . . , ejk ) : this sequence is

a directed path from s to t in G}
Pk
with cost c(P ) = i=1 cji .

Now define the node-incidence matrix A = [ai,j ],


 +1 if arc ej leaves node i
!
i = 1, . . . , |V | and

aij = −1 if arc ej enters node i
 0
 j = 1, . . . , |E|
otherwise

14
 
 +1 if arc ej leaves node i
!
i = 1, . . . , |V | and

aij = −1 if arc ej enters node i
 0
 j = 1, . . . , |E|
otherwise

a
e1 3
e4
1
2
s e3 t

2 e5
e2 1
b

 
e1 e2 e3 e4 e5
 +1 +1
s  0 0 0
 


A= t 
 0 0 0 −1 −1 

a 
 −1 0 +1 +1 0 

b 0 −1 −1 0 +1

15
To create a LP introduce fj to denote flow through arc
ej . The intuition is that we would like to send one unit
of flow from s to t. The cost of one unit of flow through
ej will be cj so a shortest s − t path “should” be one
that minimizes cost.

We need flow conservation at every non s, t node vi.

This corresponds to a′if = 0. On the other hand, one
net unit of flow leaves s and one net unit enters t so
the problem that we want to solve is

min c′f
 
+1
 −1
 
where the +1 corre-

0
 
 
Af =  
sponds to row s and the
 0 
..
 



 −1 corresponds to row t.
0
f ≥0

Note that it is possible that the fj “could” take on non-

integer values but it is easy to see that there is an
optimal solution in which each fj = 1 (in shortest
path) or fj = 0 (not in shortest path). We will prove
this more formally later when discussing unimodular
matrices.
16
 min c′f
 
+1
 −1 
 
where the +1 corre-
0
 
 
Af = 
 
sponds to row s and the
0 
.
 

 . 
 −1 corresponds to row t.
0
f ≥0

Note that the |V | equations are redundant and we

can therefore leave out any one equation. It is most
convenient to leave out the row t equation since this
will leave a nonnegative cost column in our simplex
tableau.

17
 Example

a
e1 3
e4
1
2
s e3 t

2 e5
e2 1
b

f1 f2 f3 f4 f5
−z= 0 1 2 2 3 1
1 1 1 0 0 0
0 −1 0 1 1 0
0 0 −1 −1 0 1
Now create a basis with {f1, f4, f5}.
f1 f2 f3 f4 f5
−z= −4 0 −1 0 0 0
f1 = 1 1 1 0 0 0
f4 = 1 0 1 1 1 0
f5 = 0 0 −1 −1 0 1

A basis corresponds to a set of arcs containing a

s−t path. Degenerate elements in basis are non-path
arcs.
18
 a

1111111111 111111111
0000000000 000000000
Leaving arc

0000000000 111111111
1111111111 000000000
e1
e4

0000000000 111111111
1111111111 000000000
0000000000
1111111111
0000000000
1111111111 000000000
111111111
000000000
111111111
0000000000
1111111111
s e3 t

0000000000
1111111111
0000000000
1111111111 000000000
111111111
0000000000
1111111111 000000000
111111111
e5
0000000000
1111111111
e2
0000000000
1111111111
0000000000 111111111
1111111111 000000000
Entering
arc b

f1 f2 f3 f4 f5
−z= −4 0 −1 0 0 0
f1 = 1 1 1 0 0 0
f4 = 1 0 1 1 1 0
f5 = 0 0 −1 −1 0 1

Pivoting lets us remove f4 from the basis and add f2.

Since c̄ ≥ 0 we see that solution (0, 1, 0, 0, 1) is fea-
sible optimal so path {f2, f5} is optimal.

f1 f2 f3 f4 f5
−z= −3 0 0 1 1 0
f1 = 0 1 0 −1 −1 0
f2 = 1 0 1 1 1 0
f5 = 1 0 0 0 1 1

19
 Primal

min c′f
Dual
 
+1
 −1 max πs − πt
 

0
 
π ′A ≤ c′
 
Af =  
 0 
.. π≷0
 
 
 
0
f ≥0

Since the column corresponding to e = (i, j) has a

1 in row i, a −1 in row j and a 0 elsewhere, the dual
inequalities can be written as

πi − πj ≤ cij for each (i, j) ∈ E (4)

The complementary slackness conditions then say:

Path f and assignment π are jointly optimal iff

(i) each arc e = (i, j) in shortest path, i.e.,
fe > 0 corresponds to πi − πj = cij and
(ii) πi − πj < cij corresponds to f(i,j) = 0, i.e.,
e = (i, j) not in shortest path.
20
Path f and assignment π are jointly optimal iff
(i) each arc e = (i, j) in shortest path, i.e.,
fe > 0 corresponds to πi − πj = cij and
(ii) πi − πj < cij corresponds to f(i,j) = 0, i.e.,
e = (i, j) not in shortest path.

Let the shortest s − t path found by simplex be

ek ek−1, . . . , e1,
where ei = ui, vi, uk = s, v1 = t and vi = ui−1.

If v is some node on the path then, by definition,

πv − πt is the shortest distance from v to t
so πs − πt, the objective maximized by the dual, is
exactly the shortest distance from s to t.

21
 3
a

3
1
2
s t
3 0
2
1
b 1

The figure shows an optimal π corresponding to short-

est path
(s, b), (b, t)
Note that when we calculate π we don’t really have
a value for πt, but it can be calculated from the fact
that fb,t = 1 > 0 and the complementary slackness
conditions which force

πt = πb − cb,t = 1 − 1 = 0

22
 Initial Tableau Final Tableau
cj cj πj

I B −1

Assume, WLOG, that simplex starts with Identity ma-

trix in left side, e.g., slack or artificial variables.

At end of algorithm, we have essentially multiplied ma-

trix by B −1 where B is the set of columns in original
matrix corresponding to final optimal BFS.

At optimality, cost row is

′ −1 A = c − π ′A
 
0 ≤ c̄ = c − cB B
where we have already seen that π ′ = c′B B −1 is an
optimal solution to the dual. So,
c̄j = cj − πj j = 1, . . . , m
and
πj = cj − c̄j j = 1, . . . , m
and we can read off optimal dual solution from tableau.
23
 Example

In the section on simplex we saw that the tableau on

page 13 is equivalent to the two-phase tableau

xa1 xa2 xa3 x1 x2 x3 x4 x5

−z= 0 0 0 0 1 1 1 1 1
−ξ= −8 0 0 0 − 10 −8 −3 −1 −1
xa1 = 1 1 0 0 3 2 1 0 0
xa2 = 3 0 1 0 5 1 1 1 0
xa3 = 4 0 0 1 2 5 1 0 1

This has cj = 0 in real cost row. We also saw that at

optimality this transforms to

xa1 xa2 xa3 x1 x2 x3 x4 x5

−z= − 9/2 5/2 −1 −1 3/2 0 3/2 0 0
−ξ= 0 1 1 1 0 0 0 0 0
xa1 = 1/2 1/2 0 0 3/2 1 1/2 0 0
xa2 = 5/2 − 1/2 1 0 7/2 0 1/2 1 0
xa3 = 3/2 − 5/2 0 1 − 11/2 0 −3/2 0 1

Thus, an optimal dual solution would be

π = (−5/2, 1, 1)
which is exactly what we derived on page 13.
24