Linear Programming: Chapter 5 Duality

Robert J. Vanderbei October 17, 2007

Operations Research and Financial Engineering Princeton University Princeton, NJ 08544 http://www.princeton.edu/rvdb

Resource Allocation

Recall the resource allocation problem (m = 2, n = 3): maximize c1x1 + c2x2 + c3x3 subject to a11x1 + a12x2 + a13x3 b1 a21x1 + a22x2 + a23x3 b2 x1, x2, x3 0, where cj = prot per unit of product j produced bi = units of raw material i on hand aij = units raw material i required to produce 1 unit of prod j.

Closing Up Shop
If we produce one unit less of product j, then we free up: a1j units of raw material 1 and a2j units of raw material 2. Selling these unused raw materials for y1 and y2 dollars/unit yields a1j y1 + a2j y2 dollars. Only interested if this exceeds lost prot on each product j: a1j y1 + a2j y2 cj , j = 1, 2, 3.

Consider a buyer oering to purchase our entire inventory. Subject to above constraints, buyer wants to minimize cost: minimize b1 y 1 subject to a11y1 a12y1 a13y1 + + + + b2 y 2 a21y2 a22y2 a23y2 y1 , y 2 c1 c2 c3 0 .

Duality
Every Problem:
n

maximize
j=1 n

cj xj aij xj bi
j=1

subject to

i = 1, 2, . . . , m j = 1, 2, . . . , n,

xj 0 Has a Dual:
m

minimize
i=1 m

bi y i yiaij cj
i=1

subject to

j = 1, 2, . . . , n i = 1, 2, . . . , m.

yi 0

Dual of Dual
Primal Problem:
n

maximize
j=1 n

cj x j aij xj bi
j=1

Original problem is called the primal problem.

i = 1, . . . , m j = 1, . . . , n,

subject to

xj 0

A problem is dened by its data (notation used for the variables is arbitrary).

Dual in Standard Form:

m

maximize
i=1 m

bi yi aij yi cj
i=1

subject to

j = 1, . . . , n i = 1, . . . , m.

Dual is negative transpose of primal.

yi 0

Theorem Dual of dual is primal.

Weak Duality Theorem

If (x1, x2, . . . , xn) is feasible for the primal and (y1, y2, . . . , ym) is feasible for the dual, then cj xj bi y i .
j i

Proof. cj xj
j j i

yiaij yiaij xj
ij

xj

=
i

aij xj yi
j

bi y i .
i

Gap or No Gap?
An important question: Is there a gap between the largest primal value and the smallest dual value?

Primal Values Gap Primal Values

Dual Values

Dual Values

No Gap

Answer is provided by the Strong Duality Theorem (coming later).

Simplex Method and Duality

A Primal Problem:

Its Dual:

Notes: Dual is negative transpose of primal. Primal is feasible, dual is not.

Use primal to choose pivot: x2 enters, w2 leaves. Make analogous pivot in dual: z2 leaves, y2 enters.

Second Iteration
After First Pivot:

Primal (feasible):

Dual (still not feasible):

Note: negative transpose property intact. Again, use primal to pick pivot: x3 enters, w1 leaves. Make analogous pivot in dual: z3 leaves, y1 enters.

After Second Iteration

Primal: Is optimal. Dual: Negative transpose property remains intact. Is optimal. Conclusion Simplex method applied to primal problem (two phases, if necessary), solves both the primal and the dual.

Strong Duality Theorem

Conclusion on previous slide is the essence of the strong duality theorem which we now state: Theorem. If the primal problem has an optimal solution, x = (x, x, . . . , x ), 2 n 1 then the dual also has an optimal solution,
y = (y1 , y2 , . . . , ym),

and cj x = j
j i

biyi.

Paraphrase: If primal has an optimal solution, then there is no duality gap.

Duality Gap
Four possibilities: Primal optimal, dual optimal (no gap). Primal unbounded, dual infeasible (no gap). Primal infeasible, dual unbounded (no gap). Primal infeasible, dual infeasible (innite gap). Example of innite gap: maximize subject to 2x1 x2 x1 x2 1 x1 + x2 2 x1, x2 0.

Complementary Slackness

Theorem. At optimality, we have xj zj = 0, wiyi = 0, for j = 1, 2, . . . , n, for i = 1, 2, . . . , m.

Proof

Recall the proof of the Weak Duality Theorem: cj xj

j j

(cj + zj )xj =
j i

yiaij

xj =
ij

yiaij xj

=
i

aij xj yi =
j i

(bi wi)yi
i

bi y i ,

The inequalities come from the fact that xj zj 0, wiyi 0, for all j, for all i.

By Strong Duality Theorem, the inequalities are equalities at optimality.

Dual Simplex Method

When: dual feasible, primal infeasible (i.e., pinks on the left, not on top). An Example. Showing both primal and dual dictionaries:

Looking at dual dictionary: y2 enters, z2 leaves. On the primal dictionary: w2 leaves, x2 enters. After pivot...

Dual Simplex Method: Second Pivot

Going in, we have:

Looking at dual: y1 enters, z4 leaves. Looking at primal: w1 leaves, x4 enters.

Dual Simplex Method Pivot Rule

Refering to the primal dictionary:

Pick leaving variable from those rows that are infeasible. Pick entering variable from a box with a negative value and which can be increased the least (on the dual side).

Next primal dictionary shown on next page...

Dual Simplex Method: Third Pivot

Going in, we have:

Which variable must leave and which must enter?

See next page...

Dual Simplex Method: Third PivotAnswer

Answer is: x2 leaves, x1 enters.

Resulting dictionary is OPTIMAL:

Dual-Based Phase I Method

Example:

Notes: Two objective functions: the true objective (on top), and a fake one (below it). For Phase I, use the fake objectiveits dual feasible. Two right-hand sides: the real one (on the left) and a fake (on the right). Ignore the fake right-hand sidewell use it in another algorithm later. Phase IFirst Pivot: w3 leaves, x1 enters. After rst pivot...

Dual-Based Phase I MethodSecond Pivot

Recall current dictionary:

Dual pivot: w2 leaves, x2 enters.

After pivot:

Dual-Based Phase I MethodThird Pivot

Current dictionary: Dual pivot: w1 leaves, w2 enters.

After pivot: Its feasible!

Fourth PivotPhase II

Current dictionary:

Its feasible. Ignore fake objective. Use the real thing (top row). Primal pivot: x3 enters, w4 leaves.

Final Dictionary

After pivot:

Problem is unbounded!