Stanford University — CS261: Optimization Handout 6

Luca Trevisan January 20, 2011

Lecture 6
In which we introduce the theory of duality in linear programming.

1 The Dual of Linear Program

Suppose that we have the following linear program in maximization standard form:

maximize x1 + 2x2 + x3 + x4
subject to
x1 + 2x2 + x3 ≤ 2
x2 + x4 ≤ 1
(1)
x1 + 2x3 ≤ 1
x1 ≥ 0
x2 ≥ 0
x3 ≥ 0

and that an LP-solver has found for us the solution x1 := 1, x2 := 12 , x3 := 0, x4 := 12

of cost 2.5. How can we convince ourselves, or another user, that the solution is indeed
optimal, without having to trace the steps of the computation of the algorithm?
Observe that if we have two valid inequalities

a ≤ b and c ≤ d

then we can deduce that the inequality

a+c≤b+d

(derived by “summing the left hand sides and the right hand sides” of our original
inequalities) is also true. In fact, we can also scale the inequalities by a positive
multiplicative factor before adding them up, so for every non-negative values y1 , y2 ≥ 0
we also have

1
 y1 a + y2 c ≤ y1 b + y2 d

Going back to our linear program (1), we see that if we scale the first inequality by
1
2
, add the second inequality, and then add the third inequality scaled by 21 , we get
that, for every (x1 , x2 , x3 , x4 ) that is feasible for (1),

x1 + 2x2 + 1.5x3 + x4 ≤ 2.5

And so, for every feasible (x1 , x2 , x3 , x4 ), its cost is

x1 + 2x2 + x3 + x4 ≤ x1 + 2x2 + 1.5x3 + x4 ≤ 2.5

meaning that a solution of cost 2.5 is indeed optimal.

In general, how do we find a good choice of scaling factors for the inequalities, and
what kind of upper bounds can we prove to the optimum?
Suppose that we have a maximization linear program in standard form.

maximize c1 x1 + . . . cn xn
subject to
a1,1 x1 + . . . + a1,n xn ≤ b1
..
.
(2)
am,1 x1 + . . . + am,n xn ≤ bm
x1 ≥ 0
..
.
xn ≥ 0

For every choice of non-negative scaling factors y1 , . . . , ym , we can derive the inequality

y1 · (a1,1 x1 + . . . + a1,n xn )
+···
+yn · (am,1 x1 + . . . + am,n xn )
≤ y1 b1 + · · · ym bm

which is true for every feasible solution (x1 , . . . , xn ) to the linear program (2). We
can rewrite the inequality as

(a1,1 y1 + · · · am,1 ym ) · x1
+···

2
 +(a1,n y1 · · · am,n ym ) · xn
≤ y 1 b1 + · · · y m bm

So we get that a certain linear function of the xi is always at most a certain value,
for every feasible (x1 , . . . , xn ). The trick is now to choose the yi so that the linear
function of the xi for which we get an upper bound is, in turn, an upper bound to
the cost function of (x1 , . . . , xn ). We can achieve this if we choose the yi such that

c1 ≤ a1,1 y1 + · · · am,1 ym
.. (3)
.
cn ≤ a1,n y1 · · · am,n ym

Now we see that for every non-negative (y1 , . . . , ym ) that satisfies (3), and for every
(x1 , . . . , xn ) that is feasible for (2),

c1 x1 + . . . cn xn
≤ (a1,1 y1 + · · · am,1 ym ) · x1
+···
+(a1,n y1 · · · am,n ym ) · xn
≤ y 1 b1 + · · · y m bm

Clearly, we want to find the non-negative values y1 , . . . , ym such that the above upper
bound is as strong as possible, that is we want to

minimize b1 y1 + · · · bm ym
subject to
a1,1 y1 + . . . + am,1 ym ≥ c1
..
.
(4)
an,1 y1 + . . . + am,n ym ≥ cn
y1 ≥ 0
..
.
ym ≥ 0

So we find out that if we want to find the scaling factors that give us the best possible
upper bound to the optimum of a linear program in standard maximization form, we
end up with a new linear program, in standard minimization form.

3
Definition 1 If
maximize cT x
subject to
(5)
Ax ≤ b
x≥0
is a linear program in maximization standard form, then its dual is the minimization
linear program
minimize bT y
subject to
(6)
AT y ≥ c
y≥0

So if we have a linear program in maximization linear form, which we are going to

call the primal linear program, its dual is formed by having one variable for each
constraint of the primal (not counting the non-negativity constraints of the primal
variables), and having one constraint for each variable of the primal (plus the non-
negative constraints of the dual variables); we change maximization to minimization,
we switch the roles of the coefficients of the objective function and of the right-hand
sides of the inequalities, and we take the transpose of the matrix of coefficients of the
left-hand side of the inequalities.
The optimum of the dual is now an upper bound to the optimum of the primal.
How do we do the same thing but starting from a minimization linear program?
We can rewrite

minimize cT y
subject to
Ay ≥ b
y≥0
in an equivalent way as

maximize − cT y
subject to
−Ay ≤ −b
y≥0
If we compute the dual of the above program we get

minimize − bT z
subject to
−AT z ≥ −c
z≥0

4
that is,

maximize bT z
subject to
AT z ≤ c
y≥0

So we can form the dual of a linear program in minimization normal form in the same
way in which we formed the dual in the maximization case:

• switch the type of optimization,

• introduce as many dual variables as the number of primal constraints (not

counting the non-negativity constraints),

• define as many dual constraints (not counting the non-negativity constraints)

as the number of primal variables.

• take the transpose of the matrix of coefficients of the left-hand side of the
inequality,

• switch the roles of the vector of coefficients in the objective function and the
vector of right-hand sides in the inequalities.

Note that:

Fact 2 The dual of the dual of a linear program is the linear program itself.

We have already proved the following:

Fact 3 If the primal (in maximization standard form) and the dual (in minimization
standard form) are both feasible, then

opt(primal) ≤ opt(dual)

Which we can generalize a little

Theorem 4 (Weak Duality Theorem) If LP1 is a linear program in maximiza-

tion standard form, LP2 is a linear program in minimization standard form, and LP1
and LP2 are duals of each other then:

• If LP1 is unbounded, then LP2 is infeasible;

5
 • If LP2 is unbounded, then LP1 is infeasible;

• If LP1 and LP2 are both feasible and bounded, then

opt(LP1 ) ≤ opt(LP2 )

Proof: We have proved the third statement already. Now observe that the third
statement is also saying that if LP1 and LP2 are both feasible, then they have to both
be bounded, because every feasible solution to LP2 gives a finite upper bound to the
optimum of LP1 (which then cannot be +∞) and every feasible solution to LP1 gives
a finite lower bound to the optimum of LP2 (which then cannot be −∞). 

What is surprising is that, for bounded and feasible linear programs, there is always
a dual solution that certifies the exact value of the optimum.

Theorem 5 (Strong Duality) If either LP1 or LP2 is feasible and bounded, then
so is the other, and
opt(LP1 ) = opt(LP2 )

To summarize, the following cases can arise:

• If one of LP1 or LP2 is feasible and bounded, then so is the other;

• If one of LP1 or LP2 is unbounded, then the other is infeasible;

• If one of LP1 or LP2 is infeasible, then the other cannot be feasible and bounded,
that is, the other is going to be either infeasible or unbounded. Either case can
happen.

We will return to the Strong Duality Theorem, and discuss its proof, later in the
course.