Duality in Linear Programming Standard Form for Linear Programs: Review

Consider a real-valued, unknown, n-vector x = (x1, x2, … , xn)T.

Learning Goals.
 Introduce the Dual Linear Program. A linear programming problem in standard form (A, b, c) has the three
 Widget Example and Graphical Solution. components: Constants:
A an m x n matrix,
 Basic Theory: b an m x 1 vector,
Objective Function: We wish to choose x to maximize: c an n x 1 vector.
• Mutual Bound Theorem.
cT x = c1x1 + c2x2 + … cnxn Linear function of x
• Duality Theorem. with x subject to the following constraints:

Problem Constraints: For an m x n matrix A, and an m x 1 vector b:

Ax≤b

Readings: Read text section 11.6, and sections 1 and 2 of Tom Linear inequality constraints on x
Non-negativity Constraints:
Ferguson’s notes (see course homepage).
x≥0
Notation: For two K-vectors x and y,
x ≤ y iff xk ≤ yk for each k = 1, 2, …, K.
Other inequalities (≥, etc.) defined similarly.

1 2

Widget Factory Example: Revisited Widget Factory Example: Upper Bounds

Widget problem in Standard Form, constants (A, b, c). Maximize profit: cT x, where cT = (c1, c2) = (1, 2).

Unknowns: Subject to: Ax ≤ b and x ≥ 0.

x = (x1, x2)T number (in thousands) of the two widget types.
Notice, for any feasible x and any y = (y1, y2, y3)T ≥ 0: Could choose
Objective function (profit): yT A ≥ c and y ≥ 0.
cT x = c1x1 + c2x2 = x1 + 2x2, so cT = (c1, c2) = (1, 2).

Problem Constraints: A x ≤ b

so E.g., y = (2, 0, 0)T gives yT A = (2, 2) ≥ cT. Therefore: Upper bound!

cTx ≤ yTAx ≤ yTb = 2b1 = 8, i.e. max profit cTx ≤ 8.

Non-negativity Constraints: In general, for any feasible x and any y such that
x≥0 y ≥ 0 and yT A ≥ cT, Feasible y
Dual LP
we have the inequality:
minimize yTb
cTx ≤ yTAx ≤ yTb.

3 4
 Duality in Linear Programming Duality Theorem of Linear Programming

Defn. Consider the linear programming problem (in standard form): LP Duality Theorem: Consider the linear programming problem:
maximize cT x
(1)
subject to A x ≤ b and x ≥ 0, maximize cT x
(1)
The dual of this LP problem is the LP minimization problem: subject to A x ≤ b and x ≥ 0.
minimize yT b
(2)
subject to yTA ≥ cT and y ≥ 0. The feasible set F for (1) is not empty and cT x is bounded above for
These two LP problems are said to be duals of each other. x є F iff the corresponding dual LP (2) (above) has a non-empty feasible
set G = {y | yTA ≥ cT and y ≥ 0} and yTb is bounded below for y є G.
Mutual Bound Theorem: If x is a feasible solution of LP (1) and y is a Moreover, in this case, max { cTx | x є F } = min { yTb | y є G }.
feasible solution of LP (2), then cT x ≤ yTAx ≤ yT b.
Pf: See previous slide. Note: For integer linear programming (i.e., xi, yj є Z) there can be a gap.
Gap?
No Gap!
cT x1 (y1)T b

z (profit) z (profit)
max cT x min yT b max cT x = min yT b
cT x for feasible x yT b for feasible y
cT x for x є F yT b for y є G

5 6

Vertices of Dual Linear Program Vertices of LP and Dual LP

Consider the Dual LP problem: Define the m+n dimensional binary valued indicator vector δ(s) where δj = 1
minimize yT b (2) if j є s, and δj = 0 otherwise. Define δ(t) similarly.
subject to yTA ≥ cT and y ≥ 0.

We can rewrite the feasibility conditions (2) of the dual as

m x (n + m) matrix
Vertex of LP: If the jth coefficient of δ(s) is one (i.e., [δ(s)]j = 1) then
(3)
the jth row below is an equality for vertex x:
The dual LP is an LP, and vertices can be defined the same way as we did
before.

Let t = {t1, t2, … , tm} be a selection of m columns of (3), 1 ≤ ti ≤ m+n. Vertex of Dual LP: If the ith coefficient of δ(t) is one (i.e., [δ(t)]i = 1)
Define E(t) to be the m x m matrix formed from the t-columns of D, and then the ith column below is an equality for vertex y:
eT(t) the (1 x m)-vector formed from the same columns of dT.

A point v є [Rm is a vertex of the feasible set (3) iff there exists an t
such that E(t) is nonsingular, vT = eT(t)[E(t)]-1, and v satisfies (3).

7 8
 Complementary Slackness Proposing Vertices for the Dual LP

Complementary Slackness: Given feasible solutions x and y of the LP and Spatially, complementary slackness suggests:
the dual LP, respectively. Then x and y are optimal iff

and
Where βi+n = bitFlip(αi) for i = 1, 2, …, m.
And βj = bitFlip(αj+m) for j = 1, 2, …, n.

Pf: Follows from cTx = yTAx = yTb as a necessary and sufficient condition Since sum(δ(s)) = n, length(δ(·)) = n+m, it follows sum(δ(t)) = m.
for the optimality of the feasible solutions x and y.

Suggests choosing of the sets s and t (defining the vertex x of the LP

Given a vertex x of the LP, defined by s, we can use the rule above to
and the vertex y of the dual LP) such that the bit vectors satisfy:
try to construct t and the corresponding vertex of the dual LP. We can
use the pair to check for optimality. See the following example.

9 10

Graphing the Widget Factory Example: Cont. Widget Factory Example: Optimal Dual Solution

Example: x = (x1, x2)T. Linear Program specified by (A, b, c). E.G. (Cont.): This vertex of the LP was obtained using s = {1, 3}. Generate
corresponding column selection t (possibly a feasible vertex for dual LP):
Objective Function: cTx,
c = (1, 2)T

Problem Constraints:
x2 Half-plane:
Ax ≤ b, x1 + x2 ≤ 4
Half-plane: So t = { 1, 2, 4} and we select columns 1, 2, and 4 from (3) below.
-3x1 + 10x2 ≤ 15
3- (3)

Optimal Solution:
2-
xT = (25, 27)/13

Non-negativity:
x ≥ 0.
1- Soln: yT = (16, 0, 1)/13.
Optimal Vertex: Check: cT x = (1, 2) (25, 27)T/13 = 79/13 = yTb = (16, 0, 1)/13(4, 1, 15)T
s = { 1, 3 }
Conclude: y is a feasible vertex of the dual LP, and x, y are optimal.
x1
1 2 3 4

11 12