Lecture notes 4: Duality

Vincent Conitzer

1 Introduction
Let us again consider the linear program for our original painting problem instance:
maximize 3x1 + 2x2
subject to
4x1 + 2x2 ≤ 16
x1 + 2x2 ≤ 8
x1 + x2 ≤ 5
x1 ≥ 0; x2 ≥ 0
We already know that the optimal solution is to set x1 = 3, x2 = 2, for a solution value of
9 + 4 = 13. But how do we prove that this is optimal? Originally, we proved this using a picture:
from the picture it was easy to see that the optimal solution is where the lines corresponding to
the blue and red (first and third) constraints intersect, and it is easy to show that this is at the
point x1 = 3, x2 = 2. However, this approach does not generalize to more variables—perhaps
convincing pictures can still be drawn for three variables, but how would you draw one for four
variables? Moreover, it is somewhat informal and relies on our ability to draw straight lines. Once
we introduce an algorithm that always finds the optimal solution to a linear program, we can simply
give the run of the algorithm as a proof of optimality. Still, such a proof would be quite long and
unpleasant to read.
We can use duality to give a much nicer proof of optimality. To introduce duality, let us first
pursue a more modest objective: we want to find a simple upper bound on the objective value
(revenue) that can be obtained. The lower the upper bound, the better; and if we can give an upper
bound of 13, then we will have proven that our solution x1 = 3, x2 = 2 is optimal, since it in fact
produces a revenue of 13.
Here is a very simple upper bound: we know that the blue constraint 4x1 + 2x2 ≤ 16 must hold.
Because x1 is nonnegative, we must have 3x1 + 2x2 ≤ 4x1 + 2x2 ≤ 16. Hence, we know that 16 is
an upper bound on the revenue 3x1 + 2x2 that we can obtain.
We can get a better upper bound by multiplying the red constraint x1 + x2 ≤ 5 by 3. This gives
us 3x1 + 3x2 ≤ 15, and because x2 is nonnegative, 3x1 + 2x2 ≤ 3x1 + 3x2 . So 15 is also an upper
bound.
We can also multiply constraints by fractions; moreover, we can add them to each other. If we
multiply the blue constraint by 1/2, we get 2x1 + x2 ≤ 8. If we then add this to the red constraint,
we get 3x1 + 2x2 ≤ 13. Hence, we get an upper bound of 13—the optimal solution value. We cannot
get a lower upper bound than that, since then, it would not be a lower bound, because a revenue of
13 can actually be obtained. We note that in finding this optimal upper bound, we used only the
constraints that are binding in the optimal solution—that is, they are satisfied without any slack.
This is intuitively sensible because these are the constraints that matter. We will see later on in
these lecture notes that this is a special case of something known as complementary slackness.

1
 All of the above upper bounds were obtained by multiplying the constraints with various non-
negative numbers, and then adding the results together. If we multiply by a negative number,
the inequality in the constraint flips, so that would not be helpful. Let y1 , y2 , y3 be the numbers
by which we multiply the blue, green, and red constraints, respectively. Hence, the first upper
bound that we obtained consisted of setting y1 = 1, y2 = 0, y3 = 0; the second upper bound con-
sisted of setting y1 = 0, y2 = 0, y3 = 3; and the third, optimal upper bound consisted of setting
y1 = 0.5, y2 = 0, y3 = 1.
Now, let us think about how we might find the optimal upper bound. As it turns out, we can
model this as a linear program! As we already noted, all the yi should be nonnegative. In addition,
we need to make sure that, once we add up all the multiplied constraints, the coefficient on x1 is at
least 3, because the coefficient on x1 in the objective is 3. That is, 4y1 + y2 + y3 ≥ 3. Similarly, for
x2 , we must have 2y1 + 2y2 + y3 ≥ 2. These inequalities guarantee that we will in fact have a correct
upper bound. Finally, given that we satisfy all of these inequalities, we want to minimize the upper
bound that this gives us, since smaller upper bounds are better. The upper bound that we obtain
is a linear combination of the right-hand sides of the original inequalities: 16y1 + 8y2 + 5y3 . Putting
it together, we get:
minimize 16y1 + 8y2 + 5y3
subject to
4y1 + y2 + y3 ≥ 3
2y1 + 2y2 + y3 ≥ 2
y1 ≥ 0; y2 ≥ 0; y3 ≥ 0

This linear program is the dual of the original (also called primal) linear program. Any feasible
solution to the dual corresponds to an upper bound on any solution to the primal (this is known as
the weak duality property). Also, we saw that in this case, there is a feasible solution to the dual
whose value is equal to the optimal value for the primal (and therefore, we know that this feasible
solution to the dual is in fact optimal for the dual, because if there were a better solution for the
dual, it would no longer be an upper bound for the primal). It turns out that this was no accident:
in fact, the values of the optimal solutions to the primal and dual are always equal. This property
is known as strong duality. It should be emphasized that we are only talking about linear programs
here. If we are considering a (mixed) integer program (say, a maximization problem), we can take
its linear program relaxation, and then take the dual of that. Because the optimal value of the
LP relaxation is no lower than that of the original MIP, feasible solutions to the dual will in fact
correspond to upper bounds for the orginal MIP. However, in general, the optimal value for the dual
of the LP relaxation will not be equal to the optimal value for the original MIP.
Finally, let us consider what happens if we take the dual of the dual. We have not yet shown how
to take the dual of a minimization problem. One way to do this is to convert it to maximization form
first, but it is not difficult to reason about the minimization problem directly. Again, we want to put
nonnegative weights on the constraints; let us call the weights x1 and x2 for the above minimization
problem. This time, we want to find lower bounds. For example, if we set x1 = 3, x2 = 2, we obtain
the relationship 16y1 + 8y2 + 5y3 ≥ (3 · 4 + 2 · 2)y1 + (3 · 1 + 2 · 2)y2 + (3 · 1 + 2 · 1)y3 ≥ 3 · 3 + 2 · 2 = 13.
That is, we now need to make sure that the coefficients resulting from our linear combination of
constraints are less than or equal to those in the objective, and subject to this we want to maximize
the linear combination of the right-hand side. This results in the following linear program:

2
 maximize 3x1 + 2x2
subject to
4x1 + 2x2 ≤ 16
x1 + 2x2 ≤ 8
x1 + x2 ≤ 5
x1 ≥ 0; x2 ≥ 0

This is just the primal problem again! Thus, the dual of the dual is the primal (this is true in
general).

2 The dual: general form and results

We recall the following general form of a linear program:
maximize c1 x1 + . . . + cn xn
subject to
a11 x1 + . . . + a1n xn ≤ b1
..
.
am1 x1 + . . . + amn xn ≤ bm
x1 ≥ 0; . . .; xn ≥ 0
The dual of this linear program is:
minimize b1 y1 + . . . + bm ym
subject to
a11 y1 + . . . + am1 ym ≥ c1
..
.
a1n y1 + . . . + amn ym ≥ cn
y1 ≥ 0; . . .; ym ≥ 0
Note that in the dual, we have m variables and n constraints. This is because variables in the
dual correspond to constraints in the primal, and vice versa.
We have the following results.

Theorem 1 (Weak duality) If x1 , . . . , xn and y1 , . . . , ym are feasible solutions for the primal and
dual, respectively, and the primal problem is a maximization problem, then c1 x1 + . . . + cn xn ≤
b1 y 1 + . . . + bm y m .

This is merely saying that any feasible solution to the dual gives an upper bound to any feasible
solution to the primal—which should be clear from the way we introduced the dual. The strong
duality theorem is more difficult to prove:

Theorem 2 (Strong duality) If x1 , . . . , xn and y1 , . . . , ym are optimal solutions for the primal
and dual, respectively, then c1 x1 + . . . + cn xn = b1 y1 + . . . + bm ym . Moreover, if either the primal
or the dual has an optimal solution, then so does the other.

We will give a proof of the strong duality theorem later. It should be noted that this theorem
assumes that one of the primal and dual has an optimal solution. If the primal (dual) is unbounded,
then the dual (primal) has no feasible solution—because such a feasible solution would give a bound
on the former problem, which contradicts its unboundedness. It is possible for the primal and dual
to be simultaneously infeasible.

3
 The following result refers to the slack variables of the primal and dual, which indicate “by how
much” a constraint is met. That is, the primal slack variables are given by wi = bi −ai1 x1 −. . .−ain xn ,
and the dual slack variables are given by zj = cj − a1j y1 − . . . − amj ym .

Theorem 3 (Complementary slackness) Suppose that x1 , . . . , xn and y1 , . . . , ym are feasible so-

lutions for the primal and dual, respectively, and let w1 , . . . , wm and z1 , . . . , zn be the corresponding
slacks for the primal and dual, respectively. Then x1 , . . . , xn and y1 , . . . , ym are both optimal if and
only if the following conditions hold:

1. for all i, wi yi = 0, and

2. for all j, zj xj = 0.

That is, for the optimal solutions to the primal and the dual, for any variable that is set to a
positive value in the primal (dual), the corresponding slack variable in the dual (primal) must be
set to zero. Conversely, if all of these constraints are satisfied for a pair of feasible solutions, then
these solutions must be optimal.
Using complementary slackness, we can quickly convert an optimal solution to the primal to an
optimal solution to the dual (and vice versa). Let us consider the painting problem instance again,
and suppose that somehow we know that the optimal solution to the primal is x1 = 3, x2 = 2. If
we consider the second (green) constraint x1 + 2x2 ≤ 8, we see that there is a slack of 1 on this
constraint for the optimal solution (x1 +2x2 = 3+4 = 7 = 8−1). This means that the corresponding
dual variable y2 must be zero in an optimal solution to the dual. Also, because both x1 and x2 are
set to nonzero values, the dual constraints 4y1 + y2 + y3 ≥ 3 and 2y1 + 2y2 + y3 ≥ 2 must both be
binding in an optimal solution to the dual, that is, there can be no slack on these constraints. This
leaves us to solve the system of equalities 4y1 + y3 = 3 and 2y1 + y3 = 2, which has as its unique
solution y1 = 0.5, y3 = 1.

3 Equality constraints and duals

Suppose that our linear program has an equality constraint ai1 x1 +. . .+ain xn = bi . We can still take
the dual of this linear program by simply converting it back to the standard inequality format. It is
instructive to see what happens if we do this. The equality constraint is equivalent to the following
two constraints: ai1 x1 + . . . + ain xn ≤ bi and ai1 x1 + . . . + ain xn ≥ bi . The latter can be converted
to −ai1 x1 − . . . − ain xn ≤ −bi , giving us two less-than-or-equal-to constraints. Now, in the dual,
let yi0 be the variable associated with the former constraint, and yi00 the variable associated with
the latter constraint. Then, as part of the left-hand side of the jth constraint of the dual, we have
aij yi0 −aij yi00 = aij (yi0 −yi00 ). Similarly, as part of the dual’s objective, we have bi yi0 −bi yi00 = bi (yi0 −yi00 ).
That is, yi0 and yi00 always occur together as (yi0 − yi00 ). Hence, we can just replace them with a single
new variable yi = (yi0 − yi00 ), and then all the constraints and the objective of the dual look the same
as they would have if the original constraint had been a less-than-or-equal-to constraint. There is
one difference: yi can take any negative value (by setting yi00 > yi0 ) as well as any nonnegative value
(by setting yi0 ≥ yi00 ). That is, yi is a free variable.