FOUNDATIONS OF OPTIMIZATION: IE6001

Two Phase Simplex Algorithm

Napat Rujeerapaiboon
Semester I, AY2019/2020
 Initial Basic Feasible Solution

• In STEP 0 the simplex algorithm requires an initial BFS

and the corresponding basic representation.

• One can show that finding a feasible solution is in general

as hard as finding an optimal solution!

⇒ How to construct an initial BFS?

• In general, an initial BFS can be found by using a variant of
the simplex algorithm.
• In some special cases, an initial BFS can be constructed
“manually”.
 Problems with an “All Slack Basis”

minimize x0 = c1 x1 + c2 x2 + . . . + cn xn

subject to:

a11 x1 + a12 x2 + ... + a1n xn ≤ b1

a21 x1 + a22 x2 + ... + a2n xn ≤ b2
.. .. .. ..
. . . .
am1 x1 + am2 x2 + . . . + amn xn ≤ bm

x1 ≥ 0, x2 ≥ 0, . . . , xn ≥ 0
 Problems with an “All Slack Basis”

minimize x0 = c1 x1 + c2 x2 + . . . + cn xn

subject to:

a11 x1 + a12 x2 + ... + a1n xn + xn+1 = b1

a21 x1 + a22 x2 + ... + a2n xn + xn+2 = b2
.. .. .. ..
. . . .
am1 x1 + am2 x2 + . . . + amn xn + xn+m = bm

x1 ≥ 0, . . . , xn ≥ 0, xn+1 ≥ 0, . . . , xn+m ≥ 0

⇒ This is a basic representation for I = {n + 1, . . . , n + m}.

The corresponding BS is feasible if bi ≥ 0, i = 1, . . . , m.
 Example w/o an Obvious Initial BFS

Consider a system with

• equalities,
• ”≥” inequalities and
• ”≤” inequalities,
and assume that all variables and RHS’s are nonnegative.

x1 + x2 + x3 = 10
2x1 − x2 ≥ 2
x1 − 2x2 + x3 ≤ 6

xi ≥ 0 ∀i = 1, . . . , 3
 Example w/o an Obvious Initial BFS

Standardise the system by

• adding slack variables and
• subtracting surplus variables.

x1 + x2 + x3 = 10
2x1 − x2 − x4 = 2
x1 − 2x2 + x3 + x5 = 6
xi ≥ 0 ∀i = 1, . . . , 5

⇒ No basic representation!
Only slack variables behave like basic variables!
 Example w/o an Obvious Initial BFS

Idea: Add new artificial variables to those constraints that were

originally equalities and ”≥” inequalities.

x1 + x2 + x3 + ξ1 = 10
2x1 − x2 − x4 + ξ2 = 2
x1 − 2x2 + x3 + x5 = 6

xi ≥ 0 ∀i = 1, . . . , 5, ξ1 ≥ 0, ξ2 ≥ 0.

The artificial variables behave like basic variables.

⇒ We have found a basic representation!

But this system is not equivalent to the original one!

 Example w/o an Obvious Initial BFS

Important Observation:
Any nonnegative FS (x1 , . . . , x5 , ξ1 , ξ2 ) for

x1 + x2 + x3 + ξ1 = 10
2x1 − x2 − x4 + ξ2 = 2
x1 − 2x2 + x3 + x5 = 6

with ξ1 = ξ2 = 0 provides a nonnegative FS (x1 , . . . , x5 ) for

x1 + x2 + x3 = 10
2x1 − x2 − x4 = 2
x1 − 2x2 + x3 + x5 = 6
 Example w/o an Obvious Initial BFS

To find such a solution, we solve the auxiliary LP:

minimize ζ = ξ1 + ξ2

subject to:

x1 + x2 + x3 + ξ1 = 10
2x1 − x2 − x4 + ξ2 = 2
x1 − 2x2 + x3 + x5 = 6

x1 ≥ 0, . . . , x5 ≥ 0, ξ1 ≥ 0, ξ2 ≥ 0

The initial BFS for this LP is given by ξ1 = 10, ξ2 = 2, x5 = 6

(basic variables) and x1 = · · · = x4 = 0 (nonbasic variables).
 Example w/o an Obvious Initial BFS

• To solve the auxiliary LP with the simplex algorithm, we

need a basic representation for the initial BFS.

• However, the objective function value ζ = ξ1 + ξ2 is

expressed in terms of the basic variables ξ1 and ξ2 .

• To express ζ as a function of the nonbasic variables, we

add all equations with artificial variables to the objective.

Obj. ζ − ξ1 − ξ2 = 0
+ Eq. 1 x1 + x2 + x3 + ξ1 = 10
+ Eq. 2 2x1 − x2 − x4 + ξ2 = 2
= ζ + 3x1 + x3 − x4 = 12
 Example w/o an Obvious Initial BFS

• The auxiliary LP is feasible and bounded by construction

(ζ = ξ1 + ξ2 ≥ 0 cannot drop indefinitely!).

⇒ The simplex algorithm must terminate in STEP 1 with an

optimal BFS. There are two cases:

• ζ = 0 at optimality: this implies that ξ1 = ξ2 = 0, and the

optimal BFS of the auxiliary LP provides a BFS for the
original system.

• ζ > 0 at optimality: the auxiliary LP has no feasible solution

with ξ1 = ξ2 = 0 ⇒ the original system has no BFS ⇒ it is
infeasible!
 Example w/o an Obvious Initial BFS

Solve the auxiliary LP with the simplex algorithm.

BV x1 x2 x3 x4 x5 ξ1 ξ2 RHS
ζ 3 1 −1 12
ξ1 1 1 1 1 10
ξ2 2 −1 −1 1 2
x5 1 −2 1 1 6
 Example w/o an Obvious Initial BFS

Solve the auxiliary LP with the simplex algorithm.

BV x1 x2 x3 x4 x5 ξ1 ξ2 RHS
ζ 3 1 −1 12
ξ1 1 1 1 1 10
ξ2 2 −1 −1 1 2
x5 1 −2 1 1 6
ζ 1 12 1 1
2 −1 1
2 9
1 1 1
ξ1 12 1 2 1 −2 9
x1 1 − 12 − 21 1
2 1
1 1 1
x5 −1 2 1 2 1 −2 5
 Example w/o an Obvious Initial BFS

Solve the auxiliary LP with the simplex algorithm.

BV x1 x2 x3 x4 x5 ξ1 ξ2 RHS
1 1
ζ 12 1 2 −1 12 9
1 1 1
ξ1 12 1 2 1 − 2 9
1 1 1
x1 1 −2 −2 2 1
x5 −1 12 1 1
2 1 − 1
2 5
ζ −1 −1 0
x2 1 23 1
3 − 1
3 6
1 1 1
x1 1 3 −3 3 4
x5 2 1 1 −1 14

This is a BFS for the original system!

 Two Phase Simplex: Phase 1

Step 1: Modify the constraints so that all RHS’s are

nonnegative (constraints with negative RHS × − 1).
Step 2: Identify now all equality and ≥ constraints. In Step 4
we will add artificial variables to these constraints.
Step 3: Standardise inequalities: for ≤ constraints, add slacks;
for ≥ constraints, subtract excesses.
Step 4: Add now artificial variables ξi to all ≥ or equality
constraints identified in Step 2.
Step 5: Let ζ be the sum of all artificial variables and derive
the basic representation of ζ.
Step 6: Find minimum value of ζ using the simplex algorithm.
 Two Phase Simplex: Phase 2

Case 1: ζ ∗ > 0
⇒ The original LP is infeasible.
Case 2: ζ ∗ = 0 and all ξi are nonbasic at optimality.
⇒ Remove all artificial columns from the optimal
Phase 1 tableau.
⇒ Derive the basic representation of x0 (original
objective) w.r.t. optimal index set of Phase 1.
⇒ Solve the original LP with the simplex algorithm
(Phase 2). The final basis of Phase 1 is the initial
basis of Phase 2. The optimal solution to Phase 2
is the optimal solution to the original LP.
 Two Phase Simplex: Phase 2

Case 3: ζ ∗ = 0 and at least one ξi is basic at optimality.

⇒ As ζ ∗ = 0 we conclude that all ξi = 0.
⇒ We have found a degenerate BFS for the original
problem and a basic representation for the
auxiliary problem.
⇒ As the BFS is degenerate, we can pivot on any
ypq 6= 0 corresponding to an artificial ξp and an
original variable xq without changing the BFS!
 Example (Case 2)

min x0 = 2x1 + 3x2

subject to
1
2 x1 + 41 x2 ≤ 4
x1 + 3x2 ≥ 20
x1 + x2 = 10

and

x1 , x2 ≥ 0
 Example (Case 2)

Steps 1–4 of Phase 1 transform the equality constraints to:

1
2 x1 + 14 x2 + x3 = 4
x1 + 3x2 − x 4 + ξ2 = 20
x1 + x2 + ξ3 = 10

Initial BFS for Phase 1:

Basic variables: x3 = 4, ξ2 = 20, ξ3 = 10
Nonbasic variables: x1 = x2 = x4 = 0
 Example (Case 2)

In Step 5 of Phase 1 define ζ = ξ2 + ξ3 and derive the basic

representation for ζ w.r.t. the basic variables x3 , ξ2 and ξ3 .

Row 0 ζ − ξ2 − ξ3 = 0
+ Row 2 x1 + 3x2 − x4 + ξ2 = 20
+ Row 3 x1 + x2 + ξ3 = 10
= ζ + 2x1 + 4x2 − x4 = 30

⇒ ζ = ξ2 + ξ3 = 30 − 2x1 − 4x2 + x4
 Example (Case 2)

In Step 6 of Phase 1 we solve the auxiliary LP.

minimize ζ = 30 − 2x1 − 4x2 + s4

subject to:
1
2 x1 + 14 x2 + x3 = 4
x1 + 3x2 − x4 + ξ2 = 20
x1 + x2 + ξ3 = 10

x1 , x2 , x3 , x4 , ξ2 , ξ3 ≥ 0.
 Example (Case 2)

BV x1 x2 x3 x4 ξ2 ξ3 RHS
ζ 2 4 −1 30
1 1
x3 2 4 1 4
ξ2 1 3 −1 1 20
ξ3 1 1 1 10
 Example (Case 2)

BV x1 x2 x3 x4 ξ2 ξ3 RHS
ζ 2 4 −1 30
1 1
x3 2 4 1 4
ξ2 1 3 −1 1 20
ξ3 1 1 1 10
2 1 4 10
ζ 3 3 −3 3
5 1 1 7
x3 12 1 12 − 12 3
1 1 1 20
x2 3 1 − 3 3 3
2 1
ξ3 3 3 − 31 1 10
3
 Example (Case 2)

BV x1 x2 x3 x4 ξ2 ξ3 RHS
2 1
ζ 3 3 − 34 10
3
5 1 1 7
x3 12 1 12 − 12 3
1
x2 3 1 − 13 1
3
20
3
2 1
ξ3 3 3 − 311 10
3
ζ −1 −1 0
x3 1 − 18 1 5
8 −8
1
4
x2 1 − 12 1 1
2 −2 5
1 1 3
x1 1 2 −2 2 5

ζ ∗ = 0 ⇒ Phase 1 concluded.
 Example (Case 2)

BFS found in Phase 1:

Basic variables: x3 = 14 , x2 = 5, x1 = 5
Nonbasic variable: x4 = ξ2 = ξ3 = 0

There are no artificial variables in the basis ⇒ Case 2

We can drop the columns of all artificial variables:

ξ2 and ξ3 are no longer needed!
 Example (Case 2)

In Phase 2 we first derive the basic representation of

x0 = 2x1 + 3x2 w.r.t. the basic variables x1 , x2 and x3 .

Use Rows 2 and 3 of the optimal Phase 1 tableau to eliminate

x1 and x2 from Row 0 of Phase 2 (objective x0 ).
Row 0 : x0 − 2x1 − 3x2 = 0
3
+3 × (Row 2) : 3x2 − 2 x4 = 15
+2 × (Row 3) : 2x1 + x4 = 10
1
= : x0 − 2 x4 = 25

1
⇒ x0 = 2x1 + 3x2 = 25 + x4
2
 Example (Case 2)

We now begin Phase 2 with following basic representation:

1
x0 − 2 x4 = 25
1 1
x3 − 8 x4 = 4
1
x2 − 2 x4 = 5
1
x1 + 2 x4 = 5

The corresponding BFS is optimal!

IN THIS CASE: Phase 2 requires no further pivots.

IN GENERAL: Continue with the simplex algorithm.
 Example (Case 1)

Increase b2 from 20 to 36 ⇒ LP becomes infeasible.

minimize x0 = 2x1 + 3x2

subject to:
1
2 x1 + 41 x2 ≤ 4
x1 + 3x2 ≥ 36
x1 + x2 = 10

and

x1 , x2 ≥ 0
 Example (Case 1)

After Steps 1-5 we find the auxiliary LP:

minimize ζ = ξ2 + ξ3

subject to:
1
2 x1 + 14 x2 + x3 = 4
x1 + 3x2 − x 4 + ξ2 = 36
x1 + x2 + ξ3 = 10

x1 , x2 , x3 , x4 , ξ2 , ξ3 ≥ 0
 Example (Case 1)

Initial BFS for Phase 1:

Basic variables: x3 = 4, ξ2 = 36, ξ3 = 10
Nonbasic variables: x1 = x2 = x4 = 0
Find the basic representation of ζ w.r.t. this basis.

Row 0 ζ − ξ2 − ξ3 = 0
+ Row 2 x1 + 3x2 − x4 + ξ2 = 36
+ Row 3 x1 + x2 + ξ3 = 10
= ζ + 2x1 + 4x2 − x4 = 46

⇒ ζ = ξ2 + ξ3 = 46 − 2x1 − 4x2 + x4
 Example (Case 1)

Solve the auxiliary LP with the simplex algorithm.

BV x1 x2 x3 x4 ξ2 ξ3 RHS
ζ 2 4 −1 46
1 1
x3 2 4 1 4
ξ2 1 3 −1 1 36
ξ3 1 1 1 10
 Example (Case 1)

Solve the auxiliary LP with the simplex algorithm.

BV x1 x2 x3 x4 ξ2 ξ3 RHS
ζ 2 4 −1 46
1 1
x3 2 4 1 4
ξ2 1 3 −1 1 36
ξ3 1 1 1 10
ζ −2 −1 −4 6
1
x3 4 1 − 14 3
2
ξ2 −2 −1 1 −3 6
x2 1 1 1 10

In Row 0 of Tableau 2 no variable has a positive coefficient.

⇒ Optimal Phase 1 tableau with ζ ∗ = 6 > 0 ⇒ no FS.