Math 407A: Linear Optimization

Lecture 11: The Dual Simplex Algorithm

Math Dept, University of Washington

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 1 / 16
 1 The Dual Simplex Algorithm

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 2 / 16
 The Dual Simplex Algorithm

P maximize −4x1 − 2x2 − x3

subject to −x1 − x2 + 2x3 ≤ −3
−4x1 − 2x2 + x3 ≤ −4
x1 + x2 − 4x3 ≤ 2
0 ≤ x1 , x2 , x3

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 3 / 16
 The Dual Simplex Algorithm

P maximize −4x1 − 2x2 − x3 D minimize −3y1 − 4y2 + 2y3

subject to −x1 − x2 + 2x3 ≤ −3 subject to −y1 − 4y2 + y3 ≥ −4
−4x1 − 2x2 + x3 ≤ −4 −y1 − 2y2 + y3 ≥ −2
x1 + x2 − 4x3 ≤ 2 2y1 + y2 − 4y3 ≥ −1
0 ≤ x1 , x2 , x3 0 ≤ y1 , y2 , y3

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 3 / 16
 The Dual Simplex Algorithm

P maximize −4x1 − 2x2 − x3 D minimize −3y1 − 4y2 + 2y3

subject to −x1 − x2 + 2x3 ≤ −3 subject to −y1 − 4y2 + y3 ≥ −4
−4x1 − 2x2 + x3 ≤ −4 −y1 − 2y2 + y3 ≥ −2
x1 + x2 − 4x3 ≤ 2 2y1 + y2 − 4y3 ≥ −1
0 ≤ x1 , x2 , x3 0 ≤ y1 , y2 , y3

-1 -1 2 1 0 0 -3
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 3 / 16
 The Dual Simplex Algorithm

P maximize −4x1 − 2x2 − x3 D minimize −3y1 − 4y2 + 2y3

subject to −x1 − x2 + 2x3 ≤ −3 subject to −y1 − 4y2 + y3 ≥ −4
−4x1 − 2x2 + x3 ≤ −4 −y1 − 2y2 + y3 ≥ −2
x1 + x2 − 4x3 ≤ 2 2y1 + y2 − 4y3 ≥ −1
0 ≤ x1 , x2 , x3 0 ≤ y1 , y2 , y3

-1 -1 2 1 0 0 -3
Not
-4 -2 1 0 1 0 -4
primal
1 1 -4 0 0 1 2
feasible.
-4 -2 -1 0 0 0 0

Dual feasible!
The dual has feasible origin.

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 3 / 16
 The Dual Simplex Algorithm

The tableau below is said to be dual feasible because the objective row
coefficients are all non-positive, but it is not primal feasible.

-1 -1 2 1 0 0 -3
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 4 / 16
 The Dual Simplex Algorithm

The tableau below is said to be dual feasible because the objective row
coefficients are all non-positive, but it is not primal feasible.

-1 -1 2 1 0 0 -3
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0

A tableau is optimal if and only if it is both primal feasible and dual feasible.

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 4 / 16
 The Dual Simplex Algorithm

The tableau below is said to be dual feasible because the objective row
coefficients are all non-positive, but it is not primal feasible.

-1 -1 2 1 0 0 -3
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0

A tableau is optimal if and only if it is both primal feasible and dual feasible.

Can we design a pivot for this tableau that tries to move it toward primal
feasibility while retaining dual feasibility?

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 4 / 16
 The Dual Simplex Algorithm

D
minimize −3y1 − 4y2 + 2y3
subject to −y1 − 4y2 + y3 ≥ −4
−y1 − 2y2 + y3 ≥ −2
2y1 + y2 − 4y3 ≥ −1
0 ≤ y1 , y2 , y3

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 5 / 16
 The Dual Simplex Algorithm

D
minimize −3y1 − 4y2 + 2y3 −1 −1 2 1 0 0 −3
subject to −y1 − 4y2 + y3 ≥ −4 −4 −2 1 0 1 0 −4
−y1 − 2y2 + y3 ≥ −2 1 1 −4 0 0 1 2
2y1 + y2 − 4y3 ≥ −1 −4 −2 −1 0 0 0 0
0 ≤ y1 , y2 , y3

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 5 / 16
 The Dual Simplex Algorithm

D
minimize −3y1 − 4y2 + 2y3 −1 −1 2 1 0 0 −3
dual
subject to −y1 − 4y2 + y3 ≥ −4 −4 −2 1 0 1 0 −4
objective
−y1 − 2y2 + y3 ≥ −2 1 1 −4 0 0 1 2
coefficients
2y1 + y2 − 4y3 ≥ −1 −4 −2 −1 0 0 0 0
0 ≤ y1 , y2 , y3

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 5 / 16
 The Dual Simplex Algorithm

D
minimize −3y1 − 4y2 + 2y3 −1 −1 2 1 0 0 −3
dual
subject to −y1 − 4y2 + y3 ≥ −4 −4 −2 1 0 1 0 −4
objective
−y1 − 2y2 + y3 ≥ −2 1 1 −4 0 0 1 2
coefficients
2y1 + y2 − 4y3 ≥ −1 −4 −2 −1 0 0 0 0
0 ≤ y1 , y2 , y3
↑ ↑ ↑
Dual variables
y1 y2 y3

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 5 / 16
 The Dual Simplex Algorithm

D
minimize −3y1 − 4y2 + 2y3 −1 −1 2 1 0 0 −3
dual
subject to −y1 − 4y2 + y3 ≥ −4 −4 −2 1 0 1 0 −4
objective
−y1 − 2y2 + y3 ≥ −2 1 1 −4 0 0 1 2
coefficients
2y1 + y2 − 4y3 ≥ −1 −4 −2 −1 0 0 0 0
0 ≤ y1 , y2 , y3
↑ ↑ ↑
Dual variables
y1 y2 y3

Increasing y1 decreases the value of the dual objective.

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 5 / 16
 The Dual Simplex Algorithm

Increasing y1 means we pivot on row 1.

−1 −1 2 1 0 0 −3
−4 −2 1 0 1 0 −4
1 1 −4 0 0 1 2
−4 −2 −1 0 0 0 0

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 6 / 16
 The Dual Simplex Algorithm

Increasing y1 means we pivot on row 1.

−1 −1 2 1 0 0 −3 ← pivot row
−4 −2 1 0 1 0 −4
1 1 −4 0 0 1 2
−4 −2 −1 0 0 0 0

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 6 / 16
 The Dual Simplex Algorithm

Increasing y1 means we pivot on row 1.

−1 −1 2 1 0 0 −3 ← pivot row
−4 −2 1 0 1 0 −4
1 1 −4 0 0 1 2
−4 −2 −1 0 0 0 0

By how much can we increase the value of y1 ?

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 6 / 16
 The Dual Simplex Algorithm

Increasing y1 means we pivot on row 1.

−1 −1 2 1 0 0 −3 ← pivot row
−4 −2 1 0 1 0 −4
1 1 −4 0 0 1 2
−4 −2 −1 0 0 0 0

By how much can we increase the value of y1 ?

− y1 − 4y2 + y3 ≥ −4
− y1 − 2y2 + y3 ≥ −2
2y1 + y2 − 4y3 ≥ −1

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 6 / 16
 The Dual Simplex Algorithm

Increasing y1 means we pivot on row 1.

−1 −1 2 1 0 0 −3 ← pivot row
−4 −2 1 0 1 0 −4
1 1 −4 0 0 1 2
−4 −2 −1 0 0 0 0

By how much can we increase the value of y1 ?

ratio
− y1 − 4y2 + y3 ≥ −4 4/1
− y1 − 2y2 + y3 ≥ −2 2/1
2y1 + y2 − 4y3 ≥ −1

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 6 / 16
 The Dual Simplex Algorithm

Increasing y1 means we pivot on row 1.

−1 −1 2 1 0 0 −3 ← pivot row
−4 −2 1 0 1 0 −4
1 1 −4 0 0 1 2
−4 −2 −1 0 0 0 0
4/1 2/1 ratios
By how much can we increase the value of y1 ?

ratio
− y1 − 4y2 + y3 ≥ −4 4/1
− y1 − 2y2 + y3 ≥ −2 2/1
2y1 + y2 − 4y3 ≥ −1

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 6 / 16
 The Dual Simplex Algorithm

Increasing y1 means we pivot on row 1.

−1 -1 2 1 0 0 −3 ← pivot row
−4 −2 1 0 1 0 −4
1 1 −4 0 0 1 2
−4 −2 −1 0 0 0 0
4/1 2/1 ratios
By how much can we increase the value of y1 ?

ratio
−y1 − 4y2 + y3 ≥ −4 4/1
−y1 − 2y2 + y3 ≥ −2 2/1
2y1 + y2 − 4y3 ≥ −1

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 7 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 8 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 8 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0
Any row having a negative rhs is a candidate pivot row.

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 8 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0
Any row having a negative rhs is a candidate pivot row.

Form the ratios with the negative entries in pivot row.

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 8 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0
Any row having a negative rhs is a candidate pivot row.

Form the ratios with the negative entries in pivot row.

The pivot column is given by the smallest ratio.

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 8 / 16
 The Dual Simplex Algorithm

pivot
column
↓
-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0
Any row having a negative rhs is a candidate pivot row.

Form the ratios with the negative entries in pivot row.

The pivot column is given by the smallest ratio.

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 8 / 16
 The Dual Simplex Algorithm

pivot
column
↓

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0
Any row having a negative rhs is a candidate pivot row.

Form the ratios with the negative entries in pivot row.

The pivot column is given by the smallest ratio.

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 9 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0

1 1 -2 -1 0 0 3
-2 0 -3 -2 1 0 2
0 0 -2 1 0 1 -1
-2 0 -5 -2 0 0 6

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 10 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0

1 1 -2 -1 0 0 3
-2 0 -3 -2 1 0 2
0 0 -2 1 0 1 -1 ← pivot row
-2 0 -5 -2 0 0 6

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 10 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -4 0 0 1 2
-4 -2 -1 0 0 0 0

1 1 -2 -1 0 0 3
-2 0 -3 -2 1 0 2
0 0 -2 1 0 1 -1 ← pivot row
-2 0 -5 -2 0 0 6

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 11 / 16
 The Dual Simplex Algorithm

1 1 -2 -1 0 0 3
-2 0 -3 -2 1 0 2
0 0 -2 1 0 1 -1 ← pivot row
-2 0 -5 -2 0 0 6
1 1 0 -2 0 -1 4
-2 0 0 -7/2 1 -3/2 7/2
0 0 1 -1/2 0 -1/2 1/2
-2 0 0 -9/2 0 -5/2 17/2

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 12 / 16
 The Dual Simplex Algorithm

1 1 -2 -1 0 0 3
-2 0 -3 -2 1 0 2
0 0 -2 1 0 1 -1 ← pivot row
-2 0 -5 -2 0 0 6
1 1 0 -2 0 -1 4
-2 0 0 -7/2 1 -3/2 7/2
0 0 1 -1/2 0 -1/2 1/2
-2 0 0 -9/2 0 -5/2 17/2 optimal

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 12 / 16
 The Dual Simplex Algorithm

1 1 0 -2 0 -1 4
-2 0 0 -7/2 1 -3/2 7/2
0 0 1 -1/2 0 -1/2 1/2
-2 0 0 -9/2 0 -5/2 17/2 optimal

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 13 / 16
 The Dual Simplex Algorithm

1 1 0 -2 0 -1 4
-2 0 0 -7/2 1 -3/2 7/2
0 0 1 -1/2 0 -1/2 1/2
-2 0 0 -9/2 0 -5/2 17/2 optimal
   
x1 0
 x2  =  4 
x3 1/2

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 13 / 16
 The Dual Simplex Algorithm

1 1 0 -2 0 -1 4
-2 0 0 -7/2 1 -3/2 7/2
0 0 1 -1/2 0 -1/2 1/2
-2 0 0 -9/2 0 -5/2 17/2 optimal
       
x1 0 y1 9/2
 x2  =  4  and  y2  =  0  ,
x3 1/2 y3 5/2

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 13 / 16
 The Dual Simplex Algorithm

1 1 0 -2 0 -1 4
-2 0 0 -7/2 1 -3/2 7/2
0 0 1 -1/2 0 -1/2 1/2
-2 0 0 -9/2 0 -5/2 17/2 optimal
       
x1 0 y1 9/2
 x2  =  4  and  y2  =  0  ,
x3 1/2 y3 5/2
Optimal value = −17/2.

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 13 / 16
 The Dual Simplex Algorithm

Apply the dual simplex algorithm to the following problem.

P maximize −4x1 − 2x2 − x3

subject to − x1 − x2 + 2x3 ≤ −3
−4x1 − 2x2 + x3 ≤ −4
x1 + x2 − x3 ≤ 2
0 ≤ x1 , x2 , x3 .

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 14 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -1 0 0 1 2
-4 -2 -1 0 0 0 0

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 15 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -1 0 0 1 2
-4 -2 -1 0 0 0 0
1 1 -2 -1 0 0 3
-2 0 -3 -2 1 0 2
0 0 1 1 0 1 -1
-2 0 -5 -2 0 0 6

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 15 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -1 0 0 1 2
-4 -2 -1 0 0 0 0
1 1 -2 -1 0 0 3
-2 0 -3 -2 1 0 2
0 0 1 1 0 1 -1 ← pivot row
-2 0 -5 -2 0 0 6

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 15 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -1 0 0 1 2
-4 -2 -1 0 0 0 0
1 1 -2 -1 0 0 3
-2 0 -3 -2 1 0 2
0 0 1 1 0 1 -1 ← pivot row
-2 0 -5 -2 0 0 6

No negative entry in the pivot row!

What does this mean?

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 15 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -1 0 0 1 2
-4 -2 -1 0 0 0 0
1 1 -2 -1 0 0 3
-2 0 -3 -2 1 0 2
0 0 1 1 0 1 -1 ← pivot row
-2 0 -5 -2 0 0 6

No negative entry in the pivot row!

What does this mean?

The dual problem is unbounded.

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 15 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -1 0 0 1 2
-4 -2 -1 0 0 0 0
1 1 -2 -1 0 0 3
-2 0 -3 -2 1 0 2
0 0 1 1 0 1 -1 ← pivot row
-2 0 -5 -2 0 0 6

No negative entry in the pivot row!

What does this mean?

The dual problem is unbounded.

What can you say about the primal problem?

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 15 / 16
 The Dual Simplex Algorithm

-1 -1 2 1 0 0 -3 ← pivot row
-4 -2 1 0 1 0 -4
1 1 -1 0 0 1 2
-4 -2 -1 0 0 0 0
1 1 -2 -1 0 0 3
-2 0 -3 -2 1 0 2
0 0 1 1 0 1 -1 ← pivot row
-2 0 -5 -2 0 0 6

No negative entry in the pivot row!

What does this mean?

The dual problem is unbounded.

What can you say about the primal problem?
The primal is necessarily infeasible by the Weak Duality Theorem.

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 15 / 16
 The Dual Simplex Algorithm

Solve the following LP using the dual simplex algorithm.

maximize −4x1 − 3x2 − 2x3

subject to x1 − x3 ≤ −1
−x1 − x2 ≤ −2
x1 − x2 − 2x3 ≤ 0
0 ≤ x1 , x2 , x3

Lecture 11: The Dual Simplex Algorithm (Math Dept, University Math
of Washington)
407A: Linear Optimization 16 / 16