Optimization

Chapter 2: The Simplex Method

Dr. Nguyễn Ngọc Phan

Deartment of Mathematics - Mechanics - Informatics

VNU Hanoi University of Science

Dr. Nguyễn Ngọc Phan The Simplex Method

If a linear programming problem in standard form has an
optimal solution, then there exists a basic feasible solution
that is optimal.

Dr. Nguyễn Ngọc Phan The Simplex Method

If a linear programming problem in standard form has an
optimal solution, then there exists a basic feasible solution
that is optimal.
The simplex method is based on this fact and searches for an
optimal solution by moving from one basic feasible solution to
another, along the edges of the feasible set, always in a cost
reducing direction.

Dr. Nguyễn Ngọc Phan The Simplex Method

If a linear programming problem in standard form has an
optimal solution, then there exists a basic feasible solution
that is optimal.
The simplex method is based on this fact and searches for an
optimal solution by moving from one basic feasible solution to
another, along the edges of the feasible set, always in a cost
reducing direction.
Eventually, a basic feasible solution is reached at which none
of the available edges leads to a cost reduction; such a basic
feasible solution is optimal and the algorithm terminates.

Dr. Nguyễn Ngọc Phan The Simplex Method

Consider the LP problem in standard form
n
X
f (x) = cj xj −→ min
j=1
Xn
aij xj = bi , i = 1, . . . , m
j=1

xj ≥ 0, j = 1, . . . , n

and let x 0 be the basic feasible solution associated with basis

J0 . We define ∆k , k ∈/ J0 by
X
∆k = cj xjk − ck
j∈J0

and call it the reduced cost of xk with respect to basis J0 .

Dr. Nguyễn Ngọc Phan The Simplex Method

Theorem: The optimality condition of a LP problem
For a basic feasible solution x0 associated with basis J0 of a
LP problem in standard form, if ∆k ≤ 0, ∀k ∈ / J0 then x0 is an
optimal solution.

Dr. Nguyễn Ngọc Phan The Simplex Method

Theorem.
For a basic feasible solution x0 associated with basis J0 of a
LP problem in standard form, if there exists ∆k > 0 such that
xjk ≤ 0, ∀k ∈/ J0 then the LP problem is unsolvable as the
optimal value of the objective function is −∞.

Dr. Nguyễn Ngọc Phan The Simplex Method

Theorem.
For a basic feasible solution x0 associated with basis J0 of a
LP problem in standard form, if there exists ∆k > 0 such that
xjk ≤ 0, ∀k ∈/ J0 then the LP problem is unsolvable as the
optimal value of the objective function is −∞.

Theorem
For a basic feasible solution x0 associated with basis J0 of a
LP problem in standard form, if for every ∆k > 0, there exists
xjk > 0. Then we can move from x0 to another basic feasible
solution in a cost reducing direction.

Dr. Nguyễn Ngọc Phan The Simplex Method

An iteration of the simplex method

We assume the LP problem is in standard form, and we know

a basic feasible solution x0 , associated with basis J0 consisting
of m first vectors
J0 = {A1 , . . . , Am }
From now on, Ai always means the ith column of the
constraint matrix A, and Ai is its ith row.

Dr. Nguyễn Ngọc Phan The Simplex Method

Step 1: Forming the simplex tableau

c1 c2 ... cm cm+1 ... cn

c J0 x0 x1 x2 ... xm x m+1 ... xn

c1 A1 x10 1 0 ... 0 x1,m+1 ... x1n

c2 A2 x20 0 1 ... 0 x2,m+1 ... x2n
.. .. .. .. .. .. .. .. .. ..
. . . . . . . . . .
cm Am xm0 0 0 ... 1 xm,m+1 ... xmn

f (x 0 ) 0 0 ... 0 ∆m+1 ... ∆n

Dr. Nguyễn Ngọc Phan The Simplex Method

Step 2: Check the optimality conditions

Based on the values in the last line of the simplex tableau.

If ∆k ≤ 0 ∀k ∈/ J0 then x0 is the optimal solution, and
the algorithm terminates.

Dr. Nguyễn Ngọc Phan The Simplex Method

Step 2: Check the optimality conditions

Based on the values in the last line of the simplex tableau.

If ∆k ≤ 0 ∀k ∈ / J0 then x0 is the optimal solution, and
the algorithm terminates.
If there is ∆k > 0 then x0 is not the optimal solution,
move to step 3.

Dr. Nguyễn Ngọc Phan The Simplex Method

Step 3: Check the unsolvable condition

If there is ∆k > 0 such that xjk ≤ 0 ∀j ∈ J0 then the LP

problem is not solvable as the optimal cost is −∞.

Dr. Nguyễn Ngọc Phan The Simplex Method

Step 3: Check the unsolvable condition

If there is ∆k > 0 such that xjk ≤ 0 ∀j ∈ J0 then the LP

problem is not solvable as the optimal cost is −∞.
If for every ∆k > 0, there is at least one xjk > 0 then
move to step 4.

Dr. Nguyễn Ngọc Phan The Simplex Method

Step 4: Move to better feasible solutions

Vector entering the basis: Determine max ∆k in those

∆k > 0. If max ∆k = ∆s , vector As enters the basis.

Dr. Nguyễn Ngọc Phan The Simplex Method

Step 4: Move to better feasible solutions

Vector entering the basis: Determine max ∆k in those

∆k > 0. If max ∆k = ∆s , vector As enters the basis.
Vector exiting the basis: If
xj0 xr 0
θ0 = min =
j∈J0 , xjs >0 xjs xrs

vector Ar exits the basis, xrs is called the pivot element.

Dr. Nguyễn Ngọc Phan The Simplex Method

Step 4: Move to better feasible solutions

Vector entering the basis: Determine max ∆k in those

∆k > 0. If max ∆k = ∆s , vector As enters the basis.
Vector exiting the basis: If
xj0 xr 0
θ0 = min =
j∈J0 , xjs >0 xjs xrs

vector Ar exits the basis, xrs is called the pivot element.

Add to each row of the tableau a constant multiple of the
pivot row so that the pivot element becomes one, and all
other entries in the pivot column become zero.

Dr. Nguyễn Ngọc Phan The Simplex Method

Step 4: Move to better feasible solutions

Vector entering the basis: Determine max ∆k in those

∆k > 0. If max ∆k = ∆s , vector As enters the basis.
Vector exiting the basis: If
xj0 xr 0
θ0 = min =
j∈J0 , xjs >0 xjs xrs

vector Ar exits the basis, xrs is called the pivot element.

Add to each row of the tableau a constant multiple of the
pivot row so that the pivot element becomes one, and all
other entries in the pivot column become zero.
We will get new feasible solution x 1 , better than x 0 . Return to
step 1.

Dr. Nguyễn Ngọc Phan The Simplex Method

Example. Solve the linear programming problem

f (x) = x1 − x2 + 2x3 − 2x4 − 3x6 → min

subject to


 x1 + x3 + x4 − x6 = 2

 x2 + x4 + x6 = 12


 4x3 + 2x4 + x5 + 3x6 = 9
xj ≥ 0, j = 1, . . . , 6.


Dr. Nguyễn Ngọc Phan The Simplex Method

Example. Solve the linear programming problem

f (x) = −x1 + 3x2 − 3x3 + x4 → min

subject to 

 4x1 + 3x2 − 3x3 + x4 ≥ 12

−x + x − x
1 2 3 ≤5


 x1 + 5x2 − 5x3 ≤6
xj ≥ 0, j = 1, . . . , 4.


Dr. Nguyễn Ngọc Phan The Simplex Method

The two phase method

The original simplex method requires b ≥ 0 and the matrix A

to have an identity matrix, not always straightforward. We
present here the two phase simplex method to solve LP
problems in general form.

Dr. Nguyễn Ngọc Phan The Simplex Method

The two phase method

The original simplex method requires b ≥ 0 and the matrix A

to have an identity matrix, not always straightforward. We
present here the two phase simplex method to solve LP
problems in general form.
Consider the LP problem with b ≥ 0.

f (x) = c t x → min
Ax = b
x ≥ 0.

By multiplying some constraints by −1, we can assume,

without loss of generality, that b ≥ 0.

Dr. Nguyễn Ngọc Phan The Simplex Method

We introduce the auxiliary problem

F (x, w ) = xn+1 + · · · + xn+m → min

Ax + w = b
x ≥0
w ≥0

where w = (xn+1 , . . . , xn+m ) of xn+i , with i = 1, . . . , m, are

artificial variables.

Dr. Nguyễn Ngọc Phan The Simplex Method

The auxiliary problem can also be written as

F (x, w ) = xn+1 + · · · + xn+m → min



 a11 x1 + a12 x2 + · · · + a1n xn + xn+1 = b1

a21 x1 + a22 x2 + · · · + a2n xn

+ xn+2 = b2
 .
..



a x + a x + · · · + a x
m1 1 m2 2 mn n + xn+m = bm .
We call the unit vector An+i , i = 1, . . . , m, the artificial
vectors.

Dr. Nguyễn Ngọc Phan The Simplex Method

Let X and X ′ be the feasible set of the original and auxiliary
problems, respectively.
x ∈ X if and only if (x, 0) = (x1 , . . . , xn , 0, . . . , 0) ∈ X ′ .

Dr. Nguyễn Ngọc Phan The Simplex Method

Let X and X ′ be the feasible set of the original and auxiliary
problems, respectively.
x ∈ X if and only if (x, 0) = (x1 , . . . , xn , 0, . . . , 0) ∈ X ′ .
(x, 0) ∈ X ′ is a basic feasible solution of the auxiliary
problem if and only if x ∈ X is a basic feasible solution of
the original problem.

Dr. Nguyễn Ngọc Phan The Simplex Method

Let X and X ′ be the feasible set of the original and auxiliary
problems, respectively.
x ∈ X if and only if (x, 0) = (x1 , . . . , xn , 0, . . . , 0) ∈ X ′ .
(x, 0) ∈ X ′ is a basic feasible solution of the auxiliary
problem if and only if x ∈ X is a basic feasible solution of
the original problem.
The auxiliary problem has nonempty feasible set as
(0, b) ∈ X ′ . In addition, F (x, w ) ≥ 0 for all (x, w ) ∈ X ′ .
Therefore the auxiliary problem has an optimal solution.

Dr. Nguyễn Ngọc Phan The Simplex Method

Let X and X ′ be the feasible set of the original and auxiliary
problems, respectively.
x ∈ X if and only if (x, 0) = (x1 , . . . , xn , 0, . . . , 0) ∈ X ′ .
(x, 0) ∈ X ′ is a basic feasible solution of the auxiliary
problem if and only if x ∈ X is a basic feasible solution of
the original problem.
The auxiliary problem has nonempty feasible set as
(0, b) ∈ X ′ . In addition, F (x, w ) ≥ 0 for all (x, w ) ∈ X ′ .
Therefore the auxiliary problem has an optimal solution.
Since the auxiliary problem is in standard form, it can be
solved by the simplex method to obtain the final basis
(x̄, w̄ ) = (x̄1 , . . . , x̄n+m ).

Dr. Nguyễn Ngọc Phan The Simplex Method

There are two possibilities:
1 If w̄ ̸= 0, the original problem is infeasible, and the
algorithm terminates.

Dr. Nguyễn Ngọc Phan The Simplex Method

There are two possibilities:
1 If w̄ ̸= 0, the original problem is infeasible, and the
algorithm terminates.
2 If w̄ = 0, a basic feasible solution to the original problem
has been found.

Dr. Nguyễn Ngọc Phan The Simplex Method

We consider two cases.
1 If no artificial variable is in the final basis, the artificial
variables and the corresponding columns are eliminated,
and a feasible basis for the original problem is available.

Dr. Nguyễn Ngọc Phan The Simplex Method

We consider two cases.
1 If no artificial variable is in the final basis, the artificial
variables and the corresponding columns are eliminated,
and a feasible basis for the original problem is available.
2 If there are artificial vectors in the final basis, we
eliminate columns corresponding to artificial variables no
longer in the basis. We then adjust the cost vector and
use the simplex method to solve the original problem,
using the basis feasible solution x̄.

Dr. Nguyễn Ngọc Phan The Simplex Method

Hence, to solve the original problem, we divide it into two
phases.
Phase I to figure out the basic feasible solution of the
original problem.

Dr. Nguyễn Ngọc Phan The Simplex Method

Hence, to solve the original problem, we divide it into two
phases.
Phase I to figure out the basic feasible solution of the
original problem.
Phase II to solve the original problem using the basic
feasible solution found in Phase I.

Dr. Nguyễn Ngọc Phan The Simplex Method

Hence, to solve the original problem, we divide it into two
phases.
Phase I to figure out the basic feasible solution of the
original problem.
Phase II to solve the original problem using the basic
feasible solution found in Phase I.
Be aware that:
If matrix A of the original problem already contains some
identity vectors, we just need to introduce into the
auxiliary problem a number of artificial variables just
enough to form an identity basis.

Dr. Nguyễn Ngọc Phan The Simplex Method

Hence, to solve the original problem, we divide it into two
phases.
Phase I to figure out the basic feasible solution of the
original problem.
Phase II to solve the original problem using the basic
feasible solution found in Phase I.
Be aware that:
If matrix A of the original problem already contains some
identity vectors, we just need to introduce into the
auxiliary problem a number of artificial variables just
enough to form an identity basis.
As soon as an artificial vector is driven out of the basis,
we eliminate the corresponding column in the next
tableau.

Dr. Nguyễn Ngọc Phan The Simplex Method

Example. Solve the linear programming problem by the
two–phase simplex method

f (x) = 2x1 + 6x2 − 5x3 + x4 + 4x5 → min

subject to


x1 − 4x2 + 2x3 − 5x4 + 9x5 = 3

 x2 − 3x3 + 4x4 − 5x5 = 6


 x2 − x3 + x4 − x5 = 1
xj ≥ 0, j = 1, . . . , 6.


Dr. Nguyễn Ngọc Phan The Simplex Method

Example. Solve the linear programming problem by the
two–phase simplex method

f (x) = 7x1 + x2 − 4x3 → min

subject to 

 −6x1 + 4x2 + 5x3 ≥ −20

 x + 2x + x = 8
1 2 3


 3x 1 − 2x 2 − x 3 ≤ −8
xj ≥ 0, j = 1, 2, 3.


Dr. Nguyễn Ngọc Phan The Simplex Method

Example. Solve the linear programming problem by the
two–phase simplex method

f (x) = 2x1 − 2x2 + 3x3 → min

subject to 
2x1 + 2x2 − x3 ≤ 1

x1 − x2 − 3x3 ≥ 1

xj ≥ 0, j = 1, 2, 3.


Dr. Nguyễn Ngọc Phan The Simplex Method

The big–M method

We close by mentioning an alternative approach, the big–M

method, that combines the two phases into a single one.

Dr. Nguyễn Ngọc Phan The Simplex Method

The big–M method

We close by mentioning an alternative approach, the big–M

method, that combines the two phases into a single one.
Precisely, to solve the problem

f (x) = c t x → min Ax = b x ≥ 0,

we introduce the big–M problem

F (x, w ) = c t x + M(xn+1 + · · · + xn+m ) → min

Ax + w = b, x ≥ 0, w ≥ 0,

where M is a large positive constant, w = (xn+1 , · · · , xn+m ) is

the vector of artificial variables as in Phase I simplex.

Dr. Nguyễn Ngọc Phan The Simplex Method

For a sufficiently large choice of M, if the original problem is
feasible and its optimal cost is finite, all of the artificial
variables are eventually driven to zero, which takes us back to
the minimization of the original objective function.

Dr. Nguyễn Ngọc Phan The Simplex Method

For a sufficiently large choice of M, if the original problem is
feasible and its optimal cost is finite, all of the artificial
variables are eventually driven to zero, which takes us back to
the minimization of the original objective function.
Whenever M is compared to another number, M will be
always treated as being larger.

Dr. Nguyễn Ngọc Phan The Simplex Method

Since the function F (x, w ) is linear with respect to M, then

∆j = δj M + γj .

As M is treated as being very large,

∆j < 0 if δj < 0
∆j > 0 if δj > 0
∆k > ∆j if δk > δj .

Dr. Nguyễn Ngọc Phan The Simplex Method

Example. Solve the linear programming problem using the
big–M method

f (x) = −3x1 + x2 + 3x3 − x4 → min



 x1 + 2x2 − x3 + x4 = 2

2x − 6x + 3x + 3x = 9
1 2 3 4


 x 1 − x 2 + x 3 − x 4 = 6
xj ≥ 0, j = 1, . . . , 4.


Dr. Nguyễn Ngọc Phan The Simplex Method

Example. Solve the linear programming problem using the
big–M method

f (x) = 2x1 − 2x2 + 3x3 → min


2x1 + 2x2 − x3 ≤ 1

x1 − x2 − 3x3 ≥ 1

xj ≥ 0, j = 1, 2, 3.


Dr. Nguyễn Ngọc Phan The Simplex Method