ISyE/Math/CS/Stat 525

Linear Optimization

3. The simplex method

part 1

Prof. Alberto Del Pia

University of Wisconsin-Madison

Based on the book Introduction to Linear Optimization by D. Bertsimas and J.N. Tsitsiklis
Outline

▶ From Theorem 2.7: If a LP problem in standard form has an

optimal solution, then there exists a basic feasible solution
that is optimal.
Outline

▶ From Theorem 2.7: If a LP problem in standard form has an

optimal solution, then there exists a basic feasible solution
that is optimal.
▶ The simplex method is based on this fact.
▶ It searches for an optimal solution by
moving from one basic feasible solution to
another, along the edges of the feasible
set, always in a cost reducing direction.
▶ Eventually, a basic feasible solution is
reached at which none of the available
edges leads to a cost reduction.
▶ Such a basic feasible solution is optimal
and the algorithm terminates.
Outline

Sec. 3.1 We present necessary and sufficient condition for a feasible

solution to be optimal.
Sec. 3.2 We develop the simplex method.
Sec. 1.6 We review how to count the number of operations performed
by algorithms.
Sec. 3.3 We discuss a few different implementations, including the
simplex tableau and the revised simplex method.
Outline

▶ We consider the standard form problem

minimize c′ x
subject to Ax = b
x ≥ 0.

▶ We let P be the corresponding feasible set.

▶ We assume that the dimensions of the matrix A are m × n
and that its rows are linearly independent.
▶ We continue using our previous notation:
▶ Ai is the ith column of the matrix A,
▶ a′i is the ith row of the matrix A.
3.1 Optimality conditions
Optimality conditions

Many optimization algorithms are structured as follows:

▶ Given a feasible solution, we search its neighborhood to find a
nearby feasible solution with lower cost.
▶ If no nearby feasible solution leads to a cost improvement, the
algorithm terminates and we have a locally optimal solution.
Optimality conditions

Many optimization algorithms are structured as follows:

▶ Given a feasible solution, we search its neighborhood to find a
nearby feasible solution with lower cost.
▶ If no nearby feasible solution leads to a cost improvement, the
algorithm terminates and we have a locally optimal solution.

▶ For general optimization problems, a locally optimal solution

need not be (globally) optimal.
▶ Fortunately, in LP, local optimality implies global optimality.
You will show this in an exercise.
▶ In this section, we concentrate on the problem of searching for
a direction of cost decrease in a neighborhood of a given basic
feasible solution, and on the associated optimality conditions.
Optimality conditions

▶ Suppose that we are at a point x ∈ P and that we contemplate

moving away from x, in the direction of a vector d ∈ Rn .
▶ Clearly, we should only consider those choices of d that do not
immediately take us outside the feasible set.
▶ This leads to the following definition.

Definition 3.1
Let x be a point in a polyhedron P.
A vector d ∈ Rn is said to be a
feasible direction at x, if there exists
a positive scalar θ for which
x + θd ∈ P.
Optimality conditions

▶ Let x be a basic feasible solution to the standard form

problem.
▶ Let B(1), . . . , B(m) be the indices of the basic variables, and
let
B = [AB(1) · · · AB(m) ]
be the corresponding basis matrix.
▶ In particular, we have xi = for every nonbasic variable, while
the vector xB = (xB(1) , . . . , xB(m) ) of basic variables is given by

xB =
Optimality conditions

▶ Let x be a basic feasible solution to the standard form

problem.
▶ Let B(1), . . . , B(m) be the indices of the basic variables, and
let
B = [AB(1) · · · AB(m) ]
be the corresponding basis matrix.
▶ In particular, we have xi = 0 for every nonbasic variable, while
the vector xB = (xB(1) , . . . , xB(m) ) of basic variables is given by

xB = B−1 b.
Optimality conditions

▶ We consider the possibility of moving away from x, to a new

vector
x + θd.
▶ We do so by selecting a nonbasic variable xj (which is initially
at zero level), and increasing it to a positive value θ, while
keeping the remaining nonbasic variables at zero.
▶ Algebraically, dj = 1, and di = 0 for every nonbasic index i
other than j.
▶ At the same time, the vector xB of basic variables changes to

xB + θdB ,

where dB = (dB(1) , dB(2) , . . . , dB(m) ) is the vector with those

components of d that correspond to the basic variables.
Optimality conditions
▶ Given that we are only interested in feasible solutions, we
require
A(x + θd) = b ⇔  + θAd = b
Ax

⇔ θAd = 0,
where we have used Ax = b, since x is feasible.
▶ Thus, for the equality constraints to be satisfied for θ > 0, we
need
Ad = 0.
Optimality conditions
▶ Given that we are only interested in feasible solutions, we
require
A(x + θd) = b ⇔  + θAd = b
Ax

⇔ θAd = 0,
where we have used Ax = b, since x is feasible.
▶ Thus, for the equality constraints to be satisfied for θ > 0, we
need
Ad = 0.
▶ Recall now that dj = 1, and that di = 0 for all other nonbasic
indices i.
▶ Then,
X
n X
m
0 = Ad = Ai di = AB(i) dB(i) + Aj = BdB + Aj .
i=1 i=1
▶ Since the basis matrix B is invertible, we obtain
dB = −B−1 Aj .
Optimality conditions
▶ The direction vector d that we have just constructed is
called the jth basic direction.
▶ We have so far guaranteed that the equality constraints are
respected by the vectors x + θd, for θ > 0.

n = 5, m = 3
Optimality conditions
▶ How about the nonnegativity constraints?
▶ We recall that the variable xj is increased, and all other
nonbasic variables stay at zero level.
▶ Thus, we need only worry about the basic variables. We
distinguish two cases:

n = 5, m = 3
Optimality conditions

Case (a): x is a nondegenerate basic feasible solution.

▶ Then, xB > 0, from which it

follows that, when θ is
sufficiently small,

xB + θdB ≥ 0,

and feasibility is maintained.

▶ In particular, d is a feasible
direction.
n = 5, m = 3
Optimality conditions

Case (b): x is a degenerate basic feasible solution.

▶ Then, d is not always a
feasible direction.
▶ Indeed, it is possible that,
for a basic variable,

xB(i) = 0
dB(i) < 0.

▶ In that case, for every

positive θ we have
n = 5, m = 3
xB(i) + θdB(i) < 0.
Optimality conditions
We now study the effects on the cost function if we move along a
basic direction.
▶ If d is the jth basic direction, the cost change is given by

c′ (x + θd) − c′ x = θc′ d.

▶ The rate of cost change along the direction d is given by

X
n X
m
′
cd= ci di = cB(i) dB(i) + cj = c′B dB + cj = cj − c′B B−1 Aj .
i=1 i=1

where cB = (cB(1) , . . . , cB(m) ), and where we have used

dB = −B−1 Aj .
▶ For an intuitive interpretation:
▶ cj is the cost per unit increase in the variable xj ,
▶ −c′B B−1 Aj is the cost of the compensating change in the basic
variables necessitated by the constraint Ax = b.
Optimality conditions

▶ The latter quantity is important enough to warrant a

definition.

Definition 3.2
Let x be a basic solution, let B be an associated basis matrix,
and let cB be the vector of costs of the basic variables. For
each j, we define the reduced cost c̄j of the variable xj
according to the formula

c̄j = cj − c′B B−1 Aj .

▶ Note that the definition holds also if j is a basic index!

Example 3.1
Consider the LP problem

minimize c1 x1 + c2 x2 + c3 x3 + c4 x4
subject to x1 + x2 + x3 + x4 = 2
2x1 + 3x3 + 4x4 = 2
x1 , x2 , x3 , x4 ≥ 0.

▶ We have A1 = (1, 2) and A2 = (1, 0).

▶ Since they are linearly independent, we can choose x1 and
x2 as our basic variables.
▶ The corresponding basis matrix is
 
1 1
B= .
2 0
Example 3.1
Consider the LP problem

minimize c1 x1 + c2 x2 + c3 x3 + c4 x4
subject to x1 + x2 + x3 + x4 = 2
2x1 + 3x3 + 4x4 = 2
x1 , x2 , x3 , x4 ≥ 0.

▶ We set x3 = x4 = 0, and solve for x1 , x2 , to obtain x1 = 1

and x2 = 1.
▶ We have thus obtained the nondegenerate basic feasible
solution
(1, 1, 0, 0).
Example 3.1
Consider the LP problem

minimize c1 x1 + c2 x2 + c3 x3 + c4 x4
subject to x1 + x2 + x3 + x4 = 2
2x1 + 3x3 + 4x4 = 2
x1 , x2 , x3 , x4 ≥ 0.

▶ The 3rd basic direction d is constructed as follows:

▶ We have d3 = 1 and d4 = 0.
▶ The vector dB = (d1 , d2 ) is obtained using dB = −B−1 A3 :
      
d1 0 1/2 1 −3/2
= −B−1 A3 = − = .
d2 1 −1/2 3 1/2

▶ The 3rd basic direction is then the vector d = (− 32 , 12 , 1, 0).

Example 3.1
Consider the LP problem

minimize c1 x1 + c2 x2 + c3 x3 + c4 x4
subject to x1 + x2 + x3 + x4 = 2
2x1 + 3x3 + 4x4 = 2
x1 , x2 , x3 , x4 ≥ 0.

▶ The rate of cost change along this basic direction is

3 1
c′ d = − c1 + c2 + c3 .
2 2
▶ This is the same as the reduced cost of the variable x3 .
Optimality conditions
We now calculate the reduced cost

c̄j = cj − c′B B−1 Aj .

for the case of a basic variable (j = B(i) for some i ∈ {1, . . . , m}).
▶ Since B = [AB(1) · · · AB(m) ], we have

B−1 [AB(1) · · · AB(m) ] = I,

where I is the m × m identity matrix.

▶ In particular, B−1 AB(i) is the ith column of the identity
matrix, which is the ith unit vector ei .
▶ Therefore, for every basic variable xB(i) , we have

c̄B(i) = cB(i) − c′B B−1 AB(i) = cB(i) − c′B ei = cB(i) − cB(i) = 0.

▶ Thus the reduced cost of every basic variable is zero.

Optimality conditions
▶ Our next result provides us with optimality conditions.
▶ Given our interpretation of the reduced costs as rates of cost
change along certain directions, this result is intuitive.

Theorem 3.1
Consider a basic feasible solution x associated with a basis
matrix B, and let c̄ be the corresponding vector of reduced
costs.
(a) If c̄ ≥ 0, then x is optimal.
(b) If x is optimal and nondegenerate, then c̄ ≥ 0.

Note that the contrapositive of (b) is:

(b’) If c̄j < 0 for some j, then x is degenerate or not optimal.

Let’s prove Theorem 3.1!

Optimality conditions

Theorem 3.1
Consider a basic feasible solution x associated with a basis
matrix B, and let c̄ be the corresponding vector of reduced
costs.
(a) If c̄ ≥ 0, then x is optimal.
(b) If x is optimal and nondegenerate, then c̄ ≥ 0.

▶ Note that Theorem 3.1 allows the possibility that x is a

(degenerate) optimal basic feasible solution, but that c̄j < 0
for some nonbasic index j.
▶ According to Theorem 3.1, in order to decide whether a
nondegenerate basic feasible solution is optimal, we need
only to check whether all reduced costs are nonnegative,
which is the same as examining the n − m basic directions.
Optimality conditions

Theorem 3.1
Consider a basic feasible solution x associated with a basis
matrix B, and let c̄ be the corresponding vector of reduced
costs.
(a) If c̄ ≥ 0, then x is optimal.
(b) If x is optimal and nondegenerate, then c̄ ≥ 0.

▶ If x is a degenerate basic feasible solution, an equally simple

computational test for determining whether x is optimal is
not available.
▶ Fortunately, the simplex method manages to get around
this difficulty.
Optimality conditions

▶ In order to use Theorem 3.1 and assert that a certain basic

solution is optimal, we need to satisfy two conditions:
(a) Feasibility,
(b) Nonnegativity of the reduced costs.
▶ This leads us to the following definition.

Definition 3.3
A basis matrix B is said to be optimal if:
(a) B−1 b ≥ 0, and
(b) c̄′ = c′ − c′B B−1 A ≥ 0′ .
Optimality conditions
▶ If an optimal basis is found, the corresponding basic
solution is feasible, satisfies the optimality conditions, and
is therefore optimal.
▶ On the other hand, it can happen that we have found a
basis that is not optimal, and the corresponding basic
solution is optimal.
▶ In this case at least one reduced cost is negative.
▶ Thus the basic solution is degenerate.

Definition 3.3
A basis matrix B is said to be optimal if:
(a) B−1 b ≥ 0, and
(b) c̄′ = c′ − c′B B−1 A ≥ 0′ .
3.2 Development of the simplex method
Development of the simplex method

We now continue with the development of the simplex method.

▶ Our main task is to work out the details of how to move to a

better basic feasible solution, whenever a profitable basic
direction is discovered.
▶ Let us assume that every basic feasible solution is
nondegenerate.
▶ This assumption will remain in effect until it is explicitly
relaxed later in this section.
Development of the simplex method

▶ Suppose that we are at a basic feasible solution x and that we

have computed the reduced costs c̄j of the nonbasic variables.
▶ If all of them are nonnegative, Theorem 3.1 shows that we
have an optimal solution, and we stop.
▶ Otherwise, the reduced cost c̄j of a nonbasic variable xj is
negative, and the jth basic direction d is a feasible direction of
cost decrease.
▶ While moving along this direction d, the nonbasic variable xj
becomes positive and all other nonbasic variables remain at
zero.
▶ We describe this situation by saying that Aj (or xj ) enters the
basis.
Development of the simplex method

▶ Once we start moving away from x along the direction d, we

are tracing points of the form

x + θd, where θ ≥ 0.

▶ Since costs decrease along the direction d, it is desirable to

move as far as possible.
▶ This takes us to the point x + θ∗ d, where

θ∗ = max{θ ≥ 0 | x + θd ∈ P}.

▶ The resulting cost change is

θ∗ c′ d = θ∗ c̄j .
Development of the simplex method

We now derive a formula for θ∗ .

▶ Given that Ad = 0, we have

A(x + θd) = Ax = b ∀θ ∈ R,

and the equality constraints will never be violated.

▶ Thus, x + θd can become infeasible only if one of its
components becomes negative.
Development of the simplex method

We distinguish two cases:

(a) If d ≥ 0, then x + θd ≥ 0 for all θ ≥ 0, the vector x + θd never
becomes infeasible, and we let θ∗ = ∞.
(b) If di < 0 for some i, the constraint xi + θdi ≥ 0 becomes
xi
θ≤− .
di
▶ This constraint on θ must be satisfied for every i with di < 0.
▶ Thus, the largest possible value of θ is
 
∗ xi
θ = min − .
i | di <0 di
Development of the simplex method

 
∗ xi
θ = min − .
i | di <0 di

▶ Recall that if xi is a nonbasic variable, then either xi is the

entering variable and di = 1, or else di = 0.
▶ In either case, di is nonnegative.
▶ Thus, we only need to consider the basic variables and we
have the equivalent formula
 
∗ xB(i)
θ = min − .
i=1,...,m | dB(i) <0 dB(i)

▶ Note that θ∗ > 0, because xB(i) > 0 for all i, as a consequence

of nondegeneracy.
Example 3.2
This is a continuation of Example 3.1.

minimize c1 x1 + c2 x2 + c3 x3 + c4 x4
subject to x1 + x2 + x3 + x4 = 2
2x1 + 3x3 + 4x4 = 2
x1 , x2 , x3 , x4 ≥ 0.

▶ We again consider the basic feasible solution

x = (1, 1, 0, 0).

▶ The reduced cost c̄3 of the nonbasic variable x3 was

3 1
c̄3 = − c1 + c2 + c3 .
2 2
▶ Suppose that c = (2, 0, 0, 0), in which case, we have

c̄3 = −3.
Example 3.2
▶ Since c̄3 is negative, we form the 3rd basic direction, which is
 
3 1
d = − , , 1, 0 .
2 2
▶ We consider vectors of the form

x + θd, with θ ≥ 0.
▶ As θ increases, the only component of x that decreases is the
first one (because d1 < 0).
▶ The largest possible value of θ is given by
x1 2
θ∗ = − = .
d1 3
▶ This takes us to the point
 
2 4 2
y=x+ d= 0, , , 0 .
3 3 3
Example 3.2
minimize c1 x1 + c2 x2 + c3 x3 + c4 x4
subject to x1 + x2 + x3 + x4 = 2
2x1 + 3x3 + 4x4 = 2
x1 , x2 , x3 , x4 ≥ 0.
▶ Consider our new vector y
 
4 2
y= 0, , , 0 .
3 3
▶ The columns corresponding to the nonzero variables at the
new vector y are A2 = (1, 0) and A3 = (1, 3), and are linearly
independent.
▶ Therefore, they form a basis and the vector y is the
corresponding basic feasible solution.
▶ In particular, A3 (or x3 ) has entered the basis and A1 (or x1 )
has exited the basis.
Development of the simplex method

▶ Once θ∗ is chosen, and assuming it is finite, we move to the

new feasible solution

y = x + θ∗ d.

▶ Since xj = 0 and dj = 1, we have yj = θ∗ > 0.

▶ Let ℓ be a minimizing index in the choice of θ∗ , that is,
 
xB(ℓ) xB(i)
− = min − = θ∗ .
dB(ℓ) i=1,...,m | dB(i) <0 dB(i)

▶ Hence
xB(ℓ)
yB(ℓ) = xB(ℓ) + θ∗ dB(ℓ) = xB(ℓ) −  = 0.

dB(ℓ)
d 
B(ℓ)
Development of the simplex method

▶ The basic variable xB(ℓ) has become zero, and the nonbasic
variable xj has become positive.
▶ This suggests that xj should replace xB(ℓ) in the basis.
▶ Accordingly, we take the old basis matrix B and replace AB(ℓ)
with Aj :
 
B̄ = AB(1) · · · AB(ℓ−1) Aj AB(ℓ+1) · · · AB(m) .

▶ Equivalently, we are replacing the set {B(1), . . . , B(m)} of

basic indices by a new set {B̄(1), . . . , B̄(m)} of indices given
by (
B(i), i ̸= ℓ,
B̄(i) =
j, i = ℓ.
Development of the simplex method

Theorem 3.2
(a) The columns AB̄(i) , i = 1, . . . , m, are linearly independent
and, therefore, B̄ is a basis matrix.
(b) The vector y = x + θ∗ d is a basic feasible solution
associated with the basis matrix B̄.

Let’s prove it!

Development of the simplex method

▶ Since θ∗ > 0, the new basic feasible solution x + θ∗ d is

distinct from x.
▶ Since d is a direction of cost decrease, the cost of this new
basic feasible solution is strictly smaller than the cost of x.
▶ We have therefore accomplished our objective of moving to a
new basic feasible solution with lower cost.
Development of the simplex method

▶ We can now summarize a typical iteration of the simplex

method, also known as a pivot.
▶ It is convenient to define a vector u = (u1 , . . . , um ) by letting

u = −dB = B−1 Aj ,

where Aj is the column that enters the basis.

▶ In particular,

ui = −dB(i) , for i = 1, . . . , m.
Development of the simplex method

An iteration of the simplex method

1. We start with a basis consisting of the basic columns
AB(1) , . . . , AB(m) , and an associated basic feasible solution
x.
2. Compute the reduced costs c̄j = cj − c′B B−1 Aj for all
nonbasic indices j.
▶ If they are all nonnegative, the current basic feasible
solution is optimal, and the algorithm terminates.
▶ Else, choose some j for which c̄j < 0.
3. Compute u = B−1 Aj . If no component of u is positive, we
have θ∗ = ∞, the optimal cost is −∞, and the algorithm
terminates.
Development of the simplex method

An iteration of the simplex method

4. If some component of u is positive, let
xB(i)
θ∗ = min .
i=1,...,m | ui >0 ui

5. Let ℓ be such that

xB(ℓ)
θ∗ = .
uℓ
Form a new basis by replacing AB(ℓ) with Aj . If y is the
new basic feasible solution, the values of the new basic
variables are yj = θ∗ and yB(i) = xB(i) − θ∗ ui , i ̸= ℓ.
Development of the simplex method

▶ The simplex method is initialized with an arbitrary basic

feasible solution, which, for feasible standard form problems, is
guaranteed to exist (cf. Corollary 2.2).
Development of the simplex method
▶ The following theorem states that, in the nondegenerate case,
the simplex method works correctly and terminates after a
finite number of iterations.

Theorem 3.3
Assume that the feasible set is nonempty and that every basic
feasible solution is nondegenerate. Then, the simplex method
terminates after a finite number of iterations. At termination,
there are the following two possibilities:
(a) We have an optimal basis B and an associated basic
feasible solution which is optimal.
(b) We have found a vector d satisfying Ad = 0, d ≥ 0, and
c′ d < 0, and the optimal cost is −∞.

Let’s prove it in the next slides!

Development of the simplex method

Proof (1/3):
▶ Case 1. The algorithm terminates in Step 2:
c̄j ≥ 0 for all nonbasic indices j.
▶ Then the optimality conditions in Theorem 3.1 have been
met.
▶ B is an optimal basis, and the current basic feasible solution
is optimal.
Development of the simplex method

Proof (2/3):
▶ Case 2. The algorithm terminates in Step 3:
no component of u is positive.
▶ Then we are at a basic feasible solution x and we have
discovered a nonbasic variable xj s.t. c̄j < 0 and the jth
basic direction d satisfies Ad = 0, d ≥ 0.
▶ In particular, x + θd ∈ P for all θ > 0.
▶ Since c′ d = c̄j < 0, by taking θ arbitrarily large, the cost can
be made arbitrarily negative, and the optimal cost is −∞.
Development of the simplex method

Proof (3/3):
▶ At each iteration, the algorithm moves by a positive
amount θ∗ along a direction d that satisfies c′ d < 0.
▶ Therefore, the cost of every successive basic feasible
solution visited by the algorithm is strictly less than the
cost of the previous one.
▶ In particular, no basic feasible solution can be visited twice.
▶ Since there is a finite number of basic feasible solutions
(Corollary 2.1), the algorithm must eventually
terminate.
Development of the simplex method

▶ Theorem 3.3 provides an independent proof of some of the

results of Chapter 2 for nondegenerate standard form
problems.
▶ In particular, it shows that for standard form problems that
are nondegenerate and feasible:
▶ either the optimal cost is −∞, or
▶ there exists a basic feasible solution which is optimal.
(cf. Theorem 2.8 in Section 2.6.)
The simplex method for degenerate problems
The simplex method for degenerate problems

▶ We have been working so far under the assumption that all

basic feasible solutions are nondegenerate.
▶ Suppose now that the same algorithm is used in the presence
of degeneracy.
▶ Then, the following new possibilities may be encountered in
the course of the algorithm.
The simplex method for degenerate problems
(a) If the current basic feasible solution x is degenerate, θ∗ can be
equal to zero, in which case, the new basic feasible solution y
is the same as x.
▶ This happens if some basic variable xB(ℓ) is equal to zero and
the corresponding component dB(ℓ) of the direction vector d is
negative.
The simplex method for degenerate problems
(a) If the current basic feasible solution x is degenerate, θ∗ can be
equal to zero, in which case, the new basic feasible solution y
is the same as x.
▶ This happens if some basic variable xB(ℓ) is equal to zero and
the corresponding component dB(ℓ) of the direction vector d is
negative.
▶ Nevertheless, we can still define a new basis B̄, by replacing
AB(ℓ) with Aj , and Theorem 3.2 is still valid.

Theorem 3.2
(a) The columns AB(i) , i ̸= ℓ, and Aj are linearly independent
and, therefore, B̄ is a basis matrix.
(b) The vector y = x + θ∗
d is a basic feasible solution
associated with the basis matrix B̄.
The simplex method for degenerate problems

(b) Even if θ∗ is positive, it may happen that more than one of the
original basic variables becomes zero at the new point x + θ∗ d.
▶ Since only one of them exits the basis, the others remain in
the basis at zero level, and the new basic feasible solution is
degenerate.
The simplex method for degenerate problems

▶ Basis changes while staying at the same basic feasible solution

are not in vain.
▶ A sequence of such basis changes may lead to the eventual
discovery of a cost reducing feasible direction.
The simplex method for degenerate problems

Example:

▶ The basic feasible solution x is

degenerate.
▶ If x4 and x5 are the nonbasic
variables, then
▶ the 4th basic direction is g,
▶ the 5th basic direction is f.
▶ For either of these two basic
directions, we have θ∗ = 0.
The simplex method for degenerate problems

Example: ▶ However, if we perform a

change of basis, with x4
entering the basis and x6
exiting, the new nonbasic
variables are x5 and x6 .
▶ Then
▶ the 5th basic direction is h,
▶ the 6th basic direction is −g.
▶ In particular, we can now follow
direction h to reach a new basic
feasible solution y with lower
cost.
The simplex method for degenerate problems

▶ A sequence of basis changes might lead back to the initial

basis, in which case the algorithm may loop indefinitely.
▶ This undesirable phenomenon is called cycling.
▶ It is sometimes maintained that cycling is an exceptionally
rare phenomenon.
▶ However, for many highly structured LP problems, most basic
feasible solutions are degenerate, and cycling is a real
possibility.
▶ We will see that cycling can be avoided by judiciously
choosing the variables that will enter or exit the basis.
▶ We now discuss the freedom available in this respect.
Pivot Selection
Pivot Selection

▶ The simplex algorithm, as we described it, has certain degrees

of freedom:
▶ In Step 2, we are free to choose any j whose reduced cost c̄j is
negative.
▶ In Step 5, there may be several indices ℓ that attain the
minimum in the definition of θ∗ , and we are free to choose any
one of them.
▶ Rules for making such choices are called pivoting rules.
Pivot Selection

Regarding the choice of the entering column, the following rules

are some natural candidates:
(a) Choose a column Aj , with c̄j < 0, whose reduced cost is the
most negative.
▶ Since the reduced cost is the rate of change of the cost
function, this rule chooses a direction along which the cost
decreases at the fastest rate.
Pivot Selection

Regarding the choice of the entering column, the following rules

are some natural candidates:
(a) Choose a column Aj , with c̄j < 0, whose reduced cost is the
most negative.
▶ Since the reduced cost is the rate of change of the cost
function, this rule chooses a direction along which the cost
decreases at the fastest rate.
▶ However, the actual cost decrease depends on how far we
move along the chosen direction.
▶ This suggests the next rule.
Pivot Selection

(b) Choose a column with c̄j < 0 for which the corresponding cost
decrease θ∗ |c̄j | is largest.
▶ This rule offers the possibility of reaching optimality after a
smaller number of iterations.
▶ On the other hand, the computational burden at each
iteration is larger, because we need to compute θ∗ for each
column with c̄j < 0.
▶ The available empirical evidence suggests that the overall
running time does not improve.
Pivot Selection

▶ For large problems, even rule (a) can be computationally

expensive, because it requires the computation of the reduced
cost of every variable.
▶ In practice, simpler rules are sometimes used, such as the
smallest subscript rule, that chooses the smallest j for which c̄j
is negative.
▶ Under this rule, once a negative reduced cost is discovered,
there is no reason to compute the remaining reduced costs.
Pivot Selection

▶ Regarding the choice of the exiting column, the simplest

option is again the smallest subscript rule: out of all variables
eligible to exit the basis, choose one with the smallest
subscript.
▶ It turns out that by following the smallest subscript rule for
both the entering and the exiting column, cycling can be
avoided (cf. Section 3.4).
Implementations of the simplex method

▶ In the next section, we discuss some ways of carrying out the

mechanics of the simplex method.
▶ But first, we review the conventions used in describing the
computational requirements (operation count) of algorithms.
1.6 Algorithms and operation counts
Algorithms and operation counts

▶ An algorithm is a finite set of instructions of the type used in

common programming languages.
▶ We are interested in comparing algorithms without having to
examine the details of a particular implementation.
▶ As a first approximation, this can be accomplished by counting
the number of arithmetic operations required by an algorithm.
▶ This approach is often adequate even though it ignores the
fact that adding or multiplying large integers or high-precision
floating point numbers is more demanding than adding or
multiplying single-digit integers.
Example 1.9

(a) Inner product. Given vectors a, b ∈ Rn , compute a′ b.

a′ b = a1 b1 + a2 b2 + · · · + an bn .

▶ The natural algorithm requires n multiplications and n − 1

additions.
▶ Total number of arithmetic operations: 2n − 1.
Example 1.9

(b) Matrix multiplication. Given matrices A, B of dimensions n × n,

compute C = AB.

cij = a′i Bj , ∀i, j = 1, . . . , n.

▶ There are n2 entries of AB to be evaluated.

▶ To obtain each one, the natural algorithm forms the inner
product of a row of A and a column of B.
▶ Total number of arithmetic operations: n2 (2n − 1).
Example 1.9

We estimate the rate of growth of the number of arithmetic

operations:

(a) Inner product.

▶ Total number of arithmetic operations: 2n − 1.
▶ It increases linearly with n.
Example 1.9

We estimate the rate of growth of the number of arithmetic

operations:

(a) Inner product.

▶ Total number of arithmetic operations: 2n − 1.
▶ It increases linearly with n.

(b) Matrix multiplication.

▶ Total number of arithmetic operations: n2 (2n − 1).
▶ It increases cubically with n.
Algorithms and operation counts

Definition 1.2
Let f and g be functions that map positive numbers to positive
numbers.
(a) We write f(n) = O(g(n)) if there exist positive numbers
n0 and c such that

f(n) ≤ cg(n) for all n ≥ n0 .

(b) We write f(n) = Ω(g(n)) if there exist positive numbers

n0 and c such that

f(n) ≥ cg(n) for all n ≥ n0 .

Example: 3n3 + n2 + 10 = O(n3 ), n log n = O(n2 ), n log n = Ω(n).

Algorithms and operation counts

▶ The number of operations performed by an algorithm is called

running time.
▶ Instead of trying to estimate the running time for each
possible input, it is customary to estimate the running time
for the worst possible input data in a given family.
▶ For example, if we have an algorithm for LP, we might be
interested in estimating its worst-case running time over all
problems with a given number of variables and constraints.
▶ In practice, the “average” running time of an algorithm might
be more relevant than the “worst case” running time.
However, the average running time is much more difficult to
estimate.
Example 1.10

System of linear equations. Given a matrix A of dimension n × n,

and a vector b ∈ Rn , either compute a solution or decide that no
solution exists for the system of linear equations

Ax = b.

▶ The classical method that eliminates one variable at a time

(Gaussian elimination) is known to require O(n3 ) arithmetic
operations. (Exercise!)
▶ Practical methods for matrix inversion also require O(n3 )
arithmetic operations.
Polynomial time algorithms

Is the O(n3 ) running time of Gaussian elimination good or bad?

▶ Each time that technological advances lead to computer
hardware that is faster by a factor of 23 (roughly every 3 years
by Moore’s Law), we can solve problems of twice the size than
earlier possible:
(2n)3 = 23 · n3 .
▶ A similar argument applies to algorithms whose running time
is O(nc ) for some positive constant c:
Roughly every c years we can solve problems of twice the size
than earlier possible.
▶ Such algorithms are said to run in polynomial time.
Exponential time algorithms
▶ Algorithms also exist whose running time is Ω(2n ), where n is
a parameter representing problem size; these are said to take
at least exponential time.
▶ For such algorithms, each time that computer hardware
becomes faster by a factor of 2 (roughly every year by Moore’s
Law), we can increase the value of n that we can handle only
by 1:
2n+1 = 2 · 2n .
▶ A similar argument applies to algorithms whose running time
is Ω(2cn ) for some positive constant c:
Roughly every c years we can increase the value of n that we
can handle only by 1.
▶ It is then reasonable to expect that no matter how much
technology improves, problems with truly large values of n will
always be difficult to handle.
Polynomial vs Exponential time algorithms

Example 1.11
Suppose that we have a choice of two algorithms:
▶ The running time of the first is 10n /100 (exponential).
▶ The running time of the second is 10n3 (polynomial).
Polynomial vs Exponential time algorithms

Example 1.11
Suppose that we have a choice of two algorithms:
▶ The running time of the first is 10n /100 (exponential).
▶ The running time of the second is 10n3 (polynomial).

▶ For very small n, e.g., for n = 3, the exponential time

algorithm is preferable:

103 /100 = 10 < 10 · 33 = 270.

Polynomial vs Exponential time algorithms

Example 1.11
Suppose that we have a choice of two algorithms:
▶ The running time of the first is 10n /100 (exponential).
▶ The running time of the second is 10n3 (polynomial).

▶ Suppose that we have access to a workstation that can

execute 107 arithmetic operations per second and that we
are willing to let it run for 1000 seconds (∼17 minutes).
▶ Let us figure out what size problems can each algorithm
handle within this time frame:
▶ The equation 10n /100 = 107 × 1000 yields n = 12.
▶ The equation 10n3 = 107 × 1000 yields n = 1000.
▶ Thus the polynomial time algorithm allows us to solve
much larger problems.
Algorithms and operation counts

As a first cut, it is useful to juxtapose polynomial and exponential

time algorithms:
▶ Polynomial time algorithms are viewed as fast and efficient.
▶ Exponential time algorithms are viewed as slow.
3.3 Implementations of the simplex method
Implementations of the simplex method
Let’s now get back to the simplex method.
▶ It should be clear from the statement of the algorithm that
the vectors B−1 Aj play a key role.
▶ If these vectors are available, then we can easily compute:
▶ The reduced costs

c̄j = cj − c′B B−1 Aj .

▶ The direction of motion

−u = −B−1 Aj .
▶ The stepsize
xB(i)
θ∗ = min .
i=1,...,m | ui >0 ui
▶ The main difference between alternative implementations lies
in:
▶ The way that the vectors B−1 Aj are computed,
▶ The amount of related information that is carried from one
iteration to the next.
Implementations of the simplex method

When comparing different implementations, it is important to keep

the following facts in mind (see Section 1.6).
▶ Let B be a given m × m matrix, and let b, p ∈ Rm be given
vectors.
▶ Computing the inverse of B or solving a linear system of the
form Bx = b takes O(m3 ) arithmetic operations.
▶ Computing a matrix-vector product Bb takes O(m2 )
operations.
▶ Computing an inner product p′ b takes O(m) arithmetic
operations.
Naive implementation
Naive implementation
We start by describing the most straightforward implementation.
▶ At the beginning of a typical iteration, we have the indices
B(1), . . . , B(m) of the current basic variables.
▶ We form the basis matrix B and solve the linear system
p′ B = c′B to compute

p′ = c′B B−1 .

This vector p ∈ Rm is called the vector of simplex multipliers

associated with the basis B.
Naive implementation
We start by describing the most straightforward implementation.
▶ At the beginning of a typical iteration, we have the indices
B(1), . . . , B(m) of the current basic variables.
▶ We form the basis matrix B and solve the linear system
p′ B = c′B to compute

p′ = c′B B−1 . [O(m3 ) operations]

This vector p ∈ Rm is called the vector of simplex multipliers

associated with the basis B.
Naive implementation
We start by describing the most straightforward implementation.
▶ At the beginning of a typical iteration, we have the indices
B(1), . . . , B(m) of the current basic variables.
▶ We form the basis matrix B and solve the linear system
p′ B = c′B to compute

p′ = c′B B−1 . [O(m3 ) operations]

This vector p ∈ Rm is called the vector of simplex multipliers

associated with the basis B.
▶ The reduced cost c̄j = cj − c′B B−1 Aj of any variable xj is then
obtained according to the formula

c̄j = cj − p′ Aj .

▶ Regardless of the pivoting rule employed, we may have to

compute all of the reduced costs.
Naive implementation
We start by describing the most straightforward implementation.
▶ At the beginning of a typical iteration, we have the indices
B(1), . . . , B(m) of the current basic variables.
▶ We form the basis matrix B and solve the linear system
p′ B = c′B to compute

p′ = c′B B−1 . [O(m3 ) operations]

This vector p ∈ Rm is called the vector of simplex multipliers

associated with the basis B.
▶ The reduced cost c̄j = cj − c′B B−1 Aj of any variable xj is then
obtained according to the formula

c̄j = cj − p′ Aj . [n · O(m) = O(mn) operations]

▶ Regardless of the pivoting rule employed, we may have to

compute all of the reduced costs.
Naive implementation

▶ Once a column Aj is selected to enter the basis, we solve the

linear system Bu = Aj in order to determine the vector

u = B−1 Aj .
Naive implementation

▶ Once a column Aj is selected to enter the basis, we solve the

linear system Bu = Aj in order to determine the vector

u = B−1 Aj . [O(m3 ) operations]

Naive implementation

▶ Once a column Aj is selected to enter the basis, we solve the

linear system Bu = Aj in order to determine the vector

u = B−1 Aj . [O(m3 ) operations]

▶ At this point, we can form the direction along which we will

be moving away from the current basic feasible solution.
▶ We finally determine
xB(i)
θ∗ = min
i=1,...,m | ui >0 ui

and the variable that will exit the basis, and construct the new
basic feasible solution.
Naive implementation

▶ Once a column Aj is selected to enter the basis, we solve the

linear system Bu = Aj in order to determine the vector

u = B−1 Aj . [O(m3 ) operations]

▶ At this point, we can form the direction along which we will

be moving away from the current basic feasible solution.
▶ We finally determine
xB(i)
θ∗ = min [O(m) operations]
i=1,...,m | ui >0 ui

and the variable that will exit the basis, and construct the new
basic feasible solution.
Naive implementation: running time

▶ Thus, the total computational effort per iteration is

O(m3 + mn + m3 + m) = O(m3 + mn).

▶ We will see shortly that alternative implementations require

only
O(m2 + mn)
arithmetic operations.
▶ Therefore, the naive implementation is rather inefficient.
Revised simplex method
Revised simplex method

▶ Much of the computational burden in the naive

implementation is due to the need for solving the two linear
systems of equations

p′ B = c′B and Bu = Aj .

▶ In an alternative implementation, the matrix B−1 is made

available at the beginning of each iteration, and the vectors

p′ = c′B B−1 and u = B−1 Aj

are computed by a matrix-vector multiplication.

▶ For this approach to be practical, we need an efficient method
for updating the matrix B−1 each time that we effect a
change of basis.
Revised simplex method

▶ Let
B = [AB(1) · · · AB(m) ]
be the basis matrix at the beginning of an iteration and let

B̄ = [AB(1) · · · AB(ℓ−1) Aj AB(ℓ+1) · · · AB(m) ]

be the basis matrix at the beginning of the next iteration.

▶ These two basis matrices have the same columns except that
the ℓth column AB(ℓ) has been replaced by Aj .
▶ It is then reasonable to expect that B−1 contains information
that can be exploited in the computation of B̄−1 .
Revised simplex method
IDEA:
▶ B−1 B̄ is not very different from the identity matrix.
▶ We can compute Q such that QB−1 B̄ = I.
▶ Then B̄−1 = QB−1 .

How can we do this efficiently?

▶ Since B−1 B = I, we see that B−1 AB(i) = ei . We have:
 
1 u1
 .. .. 
 . . 
 
−1 
B B̄ = [e1 · · · eℓ−1 u eℓ+1 · · · em ] =  uℓ ,

 .. .. 
 . . 
um 1

where u = B−1 Aj . (!!)

▶ Q only needs to change the above matrix to the identity.
Revised simplex method

Definition 3.4
Given a matrix, the operation of adding a constant multiple of
one row to the same or to another row is called an elementary
row operation.

▶ Performing an elementary row operation on a matrix C is

equivalent to forming the matrix QC, where Q is a suitably
constructed square matrix.
Example 3.3

Let    
1 0 2 1 2
Q = 0 1 0 , C = 3 4 ,
0 0 1 5 6
and note that
   
1+2·5 2+2·6 11 14
QC =  3 4  =  3 4 .
5 6 5 6

▶ Multiplication from the left by the matrix Q has the effect of

multiplying the third row of C by two and adding it to the
first row.
Revised simplex method

Let’s generalize Example 3.3.

▶ Multiplying the jth row by β and adding it to the ith row (for
i ̸= j) is the same as left-multiplying by the matrix

Q = I + Dij ,

where Dij is a matrix with all entries equal to zero, except for
the (i, j)th entry which is equal to β.
▶ The determinant of such a matrix Q is equal to 1 and,
therefore, Q is invertible.
Revised simplex method

▶ Suppose now that we apply a sequence of K elementary row

operations and that the kth such operation corresponds to
left-multiplication by a certain invertible matrix Qk .
▶ Then, the sequence of these elementary row operations is the
same as left-multiplication by the invertible matrix

QK QK−1 · · · Q2 Q1 .

▶ We conclude that performing a sequence of elementary row

operations on a given matrix is equivalent to left-multiplying
that matrix by a certain invertible matrix.
▶ We will now see how we can use elementary row operations to
compute of B̄−1 exploiting B−1 .
Revised simplex method
▶ Recall that, since B−1 B = I, B−1 AB(i) = ei .

B−1 B̄ = [e1 · · · eℓ−1 u eℓ+1 · · · em ]

 
1 u1
 .. . 

 . .. 

= u ℓ
,

 .. . . 
 . . 
um 1

where u = B−1 Aj . (!!)

▶ Our goal is to change the above matrix to the identity matrix
We apply the following sequence of elementary row
operations:
(a) For each i ̸= ℓ, we add the ℓth row times −ui /uℓ to the ith
row. (Recall that uℓ > 0. Why?) This replaces ui by zero.
(b) We divide the ℓth row by uℓ . Why is this an elementary row
operation? This replaces uℓ by one.
Revised simplex method

▶ This sequence of elementary row operations replaces the ℓth

column u by the ℓth unit vector eℓ .
▶ Furthermore, it is equivalent to left-multiplying B−1 B̄ by a
certain invertible matrix Q.
▶ Since the result is the identity, we have

QB−1 B̄ = I ⇒ QB−1 = B̄−1 .

▶ The last equation shows that if we apply the same sequence

of row operations to the matrix B−1 , we obtain B̄−1 .
▶ We conclude that all it takes to generate B̄−1 , is to start with
B−1 and apply the sequence of elementary row operations
described above.
▶ Total number of arithmetic operations:
Revised simplex method

▶ This sequence of elementary row operations replaces the ℓth

column u by the ℓth unit vector eℓ .
▶ Furthermore, it is equivalent to left-multiplying B−1 B̄ by a
certain invertible matrix Q.
▶ Since the result is the identity, we have

QB−1 B̄ = I ⇒ QB−1 = B̄−1 .

▶ The last equation shows that if we apply the same sequence

of row operations to the matrix B−1 , we obtain B̄−1 .
▶ We conclude that all it takes to generate B̄−1 , is to start with
B−1 and apply the sequence of elementary row operations
described above.
▶ Total number of arithmetic operations: O(m2 ).
Example 3.4
   
1 2 3 −4
B−1 = −2 3 1 , u =  2 , ℓ = 3.
4 −3 −2 2
Example 3.4
   
1 2 3 −4
B−1 = −2 3 1 , u =  2 , ℓ = 3.
4 −3 −2 2

▶ We have
 
1 0 −4
B−1 B̄ = [e1 e2 u] = 0 1 2  .
0 0 2

▶ Thus, our objective is to transform the vector u to the unit

vector e3 = (0, 0, 1).
Example 3.4
   
1 2 3 −4
B−1 = −2 3 1 , u =  2 , ℓ = 3.
4 −3 −2 2

▶ We multiply the third row by 2 and add it to the first row.

▶ We multiply the third row by −1 and add it to the second.
▶ We divide the third row by 2.
Applying the same row operations to B−1 we obtain
 
9 −4 −1
B̄−1 = −6 6 3 .
2 −1.5 −1

▶ When the matrix B−1 is updated in the manner we have

described, we obtain an implementation of the simplex
method known as the revised simplex method.
Revised simplex method

An iteration of the revised simplex method

1. We start with a basis consisting of the basic columns
AB(1) , . . . , AB(m) , an associated basic feasible solution x,
and the inverse B−1 of the basis matrix.
2. Compute the row vector p′ = c′B B−1 and then compute
the reduced costs c̄j = cj − p′ Aj .
▶ If they are all nonnegative, the current basic feasible
solution is optimal, and the algorithm terminates.
▶ Else, choose some j for which c̄j < 0.
3. Compute u = B−1 Aj . If no component of u is positive,
the optimal cost is −∞, and the algorithm terminates.
Revised simplex method

An iteration of the revised simplex method

4. If some component of u is positive, let
xB(i)
θ∗ = min .
i=1,...,m | ui >0 ui

5. Let ℓ be such that θ∗ = xB(ℓ) /uℓ . Form a new basis by

replacing AB(ℓ) with Aj . If y is the new basic feasible
solution, the values of the new basic variables are yj = θ∗
and yB(i) = xB(i) − θ∗ ui , i ̸= ℓ.
6. Form the m × (m + 1) matrix [B−1 | u]. Add to each one
of its rows a multiple of the ℓth row to make the last
column equal to the unit vector eℓ . The first m columns
of the result is the matrix B̄−1 .
Revised simplex method: implementation

▶ At the beginning of a typical iteration, we have the indices

B(1), . . . , B(m) of the current basic variables, and the inverse
B−1 of the basis matrix.
▶ We compute

p′ = c′B B−1 .
Revised simplex method: implementation

▶ At the beginning of a typical iteration, we have the indices

B(1), . . . , B(m) of the current basic variables, and the inverse
B−1 of the basis matrix.
▶ We compute

p′ = c′B B−1 . [O(m2 ) operations]

Revised simplex method: implementation

▶ At the beginning of a typical iteration, we have the indices

B(1), . . . , B(m) of the current basic variables, and the inverse
B−1 of the basis matrix.
▶ We compute

p′ = c′B B−1 . [O(m2 ) operations]

▶ The reduced cost c̄j = cj − c′B B−1 Aj of any variable xj is then

obtained according to the formula

c̄j = cj − p′ Aj .

▶ Regardless of the pivoting rule employed, we may have to

compute all of the reduced costs.
Revised simplex method: implementation

▶ At the beginning of a typical iteration, we have the indices

B(1), . . . , B(m) of the current basic variables, and the inverse
B−1 of the basis matrix.
▶ We compute

p′ = c′B B−1 . [O(m2 ) operations]

▶ The reduced cost c̄j = cj − c′B B−1 Aj of any variable xj is then

obtained according to the formula

c̄j = cj − p′ Aj . [O(mn) operations]

▶ Regardless of the pivoting rule employed, we may have to

compute all of the reduced costs.
Revised simplex method: implementation

▶ Once a column Aj is selected to enter the basis, we compute

the vector

u = B−1 Aj .
Revised simplex method: implementation

▶ Once a column Aj is selected to enter the basis, we compute

the vector

u = B−1 Aj . [O(m2 ) operations]

Revised simplex method: implementation

▶ Once a column Aj is selected to enter the basis, we compute

the vector

u = B−1 Aj . [O(m2 ) operations]

▶ We determine
xB(i)
θ∗ = min
i=1,...,m | ui >0 ui

and the variable that will exit the basis, and construct the new
basic feasible solution, and the new basis matrix B̄.
Revised simplex method: implementation

▶ Once a column Aj is selected to enter the basis, we compute

the vector

u = B−1 Aj . [O(m2 ) operations]

▶ We determine
xB(i)
θ∗ = min [O(m) operations]
i=1,...,m | ui >0 ui

and the variable that will exit the basis, and construct the new
basic feasible solution, and the new basis matrix B̄.
Revised simplex method: implementation

▶ Once a column Aj is selected to enter the basis, we compute

the vector

u = B−1 Aj . [O(m2 ) operations]

▶ We determine
xB(i)
θ∗ = min [O(m) operations]
i=1,...,m | ui >0 ui

and the variable that will exit the basis, and construct the new
basic feasible solution, and the new basis matrix B̄.
▶ We construct the inverse B̄−1 of B̄.
Revised simplex method: implementation

▶ Once a column Aj is selected to enter the basis, we compute

the vector

u = B−1 Aj . [O(m2 ) operations]

▶ We determine
xB(i)
θ∗ = min [O(m) operations]
i=1,...,m | ui >0 ui

and the variable that will exit the basis, and construct the new
basic feasible solution, and the new basis matrix B̄.
▶ We construct the inverse B̄−1 of B̄. [O(m2 ) operations]
Revised simplex method: running time

▶ Thus, the total number of operations per iteration is

O(m2 + mn + m2 + m + m2 ) = O(m2 + mn) = O(mn).

▶ Therefore, the revised simplex method is more efficient than

the naive implementation, which required

O(m3 + mn)

arithmetic operations.
The full tableau implementation
The full tableau implementation

We now describe the implementation of the simplex method in

terms of the so-called full tableau.
▶ Here, instead of maintaining and updating the matrix B−1 , we
maintain and update the m × (n + 1) matrix

B−1 [b | A]

with columns B−1 b, B−1 A1 , . . . , B−1 An .

▶ This matrix is called the simplex tableau.
The full tableau implementation

▶ The column B−1 b is called the zeroth column and contains

the values of the basic variables.
▶ The column B−1 Ai is called the ith column of the tableau.
▶ The column u = B−1 Aj corresponding to the variable that
enters the basis is called the pivot column.

xB(1) (B−1 A1 )1 ... u1 ... (B−1 An )1

.. .. .. ..
. . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... (B−1 An )ℓ
.. .. .. ..
. . . .
xB(m) (B−1 A1 )m . . . um . . . (B−1 An )m
The full tableau implementation

▶ If the ℓth basic variable exits the basis, the ℓth row of the
tableau is called the pivot row.
▶ The element belonging to both the pivot row and the pivot
column is called the pivot element.
▶ Note that the pivot element is uℓ and is always positive
(unless u ≤ 0, in which case the algorithm has met the
termination condition in Step 3).

xB(1) (B−1 A1 )1 ... u1 ... (B−1 An )1

.. .. .. ..
. . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... (B−1 An )ℓ
.. .. .. ..
. . . .
xB(m) (B−1 A1 )m . . . um . . . (B−1 An )m
The full tableau implementation

The information contained in the rows of the tableau

B−1 [b | A]

admits the following interpretation.

▶ The equality constraints are initially given to us in the form

b = Ax.

▶ Given the current basis matrix B, these equality constraints

can also be expressed in the equivalent form

B−1 b = B−1 Ax.

▶ The tableau provides us with the coefficients of these equality

constraints.
The full tableau implementation

▶ At the end of each iteration, we need to update the tableau

B−1 [b | A] and compute

B̄−1 [b | A].

▶ This can be accomplished by left-multiplying the simplex

tableau with a matrix Q satisfying QB−1 = B̄−1 .
▶ As explained earlier, this is the same as performing those
elementary row operations that turn B−1 to B̄−1 .
▶ That is, we add to each row a multiple of the pivot row to set
all entries of the pivot column to zero, with the exception of
the pivot element which is set to one.
The full tableau implementation

Regarding the determination of the exiting column AB(ℓ) and the

stepsize θ∗ , Steps 4 and 5 of the simplex method amount to:
▶ xB(i)
u is the ratio of the ith entry in the zeroth column of the
i
tableau to the ith entry in the pivot column of the tableau.
▶ We only consider those i for which ui is positive.
▶ The smallest ratio is equal to θ∗ and determines ℓ.

xB(1) (B−1 A1 )1 ... u1 ... (B−1 An )1

.. .. .. ..
. . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... (B−1 An )ℓ
.. .. .. ..
. . . .
xB(m) (B−1 A1 )m . . . um . . . (B−1 An )m
The zeroth row
It is customary to augment the simplex tableau by including a
top row, to be referred to as the zeroth row.

xB(1) (B−1 A1 )1 ... u1 ... (B−1 An )1

.. .. .. ..
. . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... (B−1 An )ℓ
.. .. .. ..
. . . .
xB(m) (B−1 A1 )m ... um ... (B−1 An )m
The zeroth row
It is customary to augment the simplex tableau by including a
top row, to be referred to as the zeroth row.
▶ The entry at the top left corner contains the negative of the
current cost:
−c′B xB = −c′B B−1 b.
▶ The reason for the minus sign is that it allows for a simple
update rule.

− c′B xB
xB(1) (B−1 A1 )1 ... u1 ... (B−1 An )1
.. .. .. ..
. . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... (B−1 An )ℓ
.. .. .. ..
. . . .
xB(m) (B−1 A1 )m ... um ... (B−1 An )m
The zeroth row
It is customary to augment the simplex tableau by including a
top row, to be referred to as the zeroth row.
▶ The rest of the zeroth row is the row vector of reduced
costs, that is, the vector

c̄′ = c′ − c′B B−1 A.

− c′B xB c̄1 ... c̄j ... c̄n

xB(1) (B−1 A1 )1 ... u1 ... (B−1 An )1
.. .. .. ..
. . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... (B−1 An )ℓ
.. .. .. ..
. . . .
xB(m) (B−1 A1 )m ... um ... (B−1 An )m
The zeroth row

▶ The rule for updating the zeroth row turns out to be identical
to the rule used for the other rows of the tableau:
Add a multiple of the pivot row to the zeroth row to set the
reduced cost of the entering variable to zero.
▶ We will now verify that this update rule produces the correct
results for the zeroth row.
The zeroth row
▶ At the beginning of a typical iteration, the zeroth row is of the
form
[−c′B B−1 b | c′ − c′B B−1 A] = [0 | c′ ] − c′B B−1 [b | A].
| {z }
a row vector
▶ Hence, it is equal to [0 | c′ ]
plus a linear combination of the
rows of [b | A].
▶ Let column j be the pivot column, and row ℓ be the pivot row.
▶ Note that the pivot row is of the form
h′ [b | A],
where the vector h′ is the ℓth row of B−1 .
▶ Hence, after a multiple of the pivot row is added to the zeroth
row, that row is again equal to [0 | c′ ] plus a (different) linear
combination of the rows of [b | A], and is of the form
[0 | c′ ] − p′ [b | A],
for some vector p.
The zeroth row

Beginning of iteration: [0 | c′ ] − c′B B−1 [b | A]

End of iteration: [0 | c′ ] − p′ [b | A]

▶ We now calculate the vector p using our update rule.

▶ We should obtain
p′ = c′B̄ B̄−1 .
The zeroth row
a) Consider the column B̄(ℓ) of the tableau.

−c′B xB c̄1 ... c̄j ... c̄n

xB(1) (B−1 A1 )1 ... u1 ... (B−1 An )1
.. .. .. ..
. . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... (B−1 An )ℓ
.. .. .. ..
. . . .
xB(m) (B−1 A1 )m . . . um . . . (B−1 An )m

▶ Recall that B̄(ℓ) = j, thus this is the pivot column.

▶ Our update rule is such that the pivot column entry of the
zeroth row becomes zero.
▶ We obtain
cB̄(ℓ) − p′ AB̄(ℓ) = 0.
The zeroth row
b) Consider the column B̄(i) of the tableau, for i ̸= ℓ.
−c′B xB c̄1 ... c̄j ... c̄B̄(i) ... c̄n
xB(1) (B−1 A1 )1 ... u1 ... (B−1 AB̄(i) )1 ... (B−1 An )1
.. .. .. .. ..
. . . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... (B−1 AB̄(i) )ℓ ... (B−1 An )ℓ
.. .. .. .. ..
. . . . .
xB(i) (B−1 A1 )i ... ui ... (B−1 AB̄(i) )i ... (B−1 An )i
.. .. .. .. ..
. . . . .
xB(m) (B−1 A1 )m . . . um . . . (B−1 AB̄(i) )m . . . (B−1 An )m

▶ This is a column corresponding to a basic variable that

stays in the basis. Thus B̄(i) = B(i).
▶ The zeroth row entry of that column is zero, before the
change of basis, since it is the reduced cost of a basic
variable.
The zeroth row
b) Consider the column B̄(i) of the tableau, for i ̸= ℓ.
−c′B xB c̄1 ... c̄j ... c̄B(i) ... c̄n
xB(1) (B−1 A1 )1 ... u1 ... (B−1 AB(i) )1 ... (B−1 An )1
.. .. .. .. ..
. . . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... (B−1 AB(i) )ℓ ... (B−1 An )ℓ
.. .. .. .. ..
. . . . .
xB(i) (B−1 A1 )i ... ui ... (B−1 AB(i) )i ... (B−1 An )i
.. .. .. .. ..
. . . . .
xB(m) (B−1 A1 )m . . . um . . . (B−1 AB(i) )m . . . (B−1 An )m

▶ This is a column corresponding to a basic variable that

stays in the basis. Thus B̄(i) = B(i).
▶ The zeroth row entry of that column is zero, before the
change of basis, since it is the reduced cost of a basic
variable.
The zeroth row
b) Consider the column B̄(i) of the tableau, for i ̸= ℓ.
−c′B xB c̄1 ... c̄j ... 0 ... c̄n
xB(1) (B−1 A1 )1 ... u1 ... (B−1 AB(i) )1 ... (B−1 An )1
.. .. .. .. ..
. . . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... (B−1 AB(i) )ℓ ... (B−1 An )ℓ
.. .. .. .. ..
. . . . .
xB(i) (B−1 A1 )i ... ui ... (B−1 AB(i) )i ... (B−1 An )i
.. .. .. .. ..
. . . . .
xB(m) (B−1 A1 )m . . . um . . . (B−1 AB(i) )m . . . (B−1 An )m

▶ This is a column corresponding to a basic variable that

stays in the basis. Thus B̄(i) = B(i).
▶ The zeroth row entry of that column is zero, before the
change of basis, since it is the reduced cost of a basic
variable.
The zeroth row
b) Consider the column B̄(i) of the tableau, for i ̸= ℓ.
−c′B xB c̄1 ... c̄j ... 0 ... c̄n
xB(1) (B−1 A1 )1 ... u1 ... (B−1 AB(i) )1 ... (B−1 An )1
.. .. .. .. ..
. . . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... (B−1 AB(i) )ℓ ... (B−1 An )ℓ
.. .. .. .. ..
. . . . .
xB(i) (B−1 A1 )i ... ui ... (B−1 AB(i) )i ... (B−1 An )i
.. .. .. .. ..
. . . . .
xB(m) (B−1 A1 )m . . . um . . . (B−1 AB(i) )m . . . (B−1 An )m

▶ Before the operation, the B(i)th column is B−1 AB(i) , thus it

is the ith unit vector.
▶ Since i ̸= ℓ, the entry in the pivot row for that column is
equal to zero.
The zeroth row
b) Consider the column B̄(i) of the tableau, for i ̸= ℓ.
−c′B xB c̄1 ... c̄j ... 0 ... c̄n
xB(1) (B−1 A1 )1 ... u1 ... 0 ... (B−1 An )1
.. .. .. .. ..
. . . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... 0 ... (B−1 An )ℓ
.. .. .. .. ..
. . . . .
xB(i) (B−1 A1 )i ... ui ... 1 ... (B−1 An )i
.. .. .. .. ..
. . . . .
xB(m) (B−1 A1 )m . . . um . . . 0 . . . (B−1 An )m

▶ Before the operation, the B(i)th column is B−1 AB(i) , thus it

is the ith unit vector.
▶ Since i ̸= ℓ, the entry in the pivot row for that column is
equal to zero.
The zeroth row
b) Consider the column B̄(i) of the tableau, for i ̸= ℓ.
−c′B xB c̄1 ... c̄j ... 0 ... c̄n
xB(1) (B−1 A1 )1 ... u1 ... 0 ... (B−1 An )1
.. .. .. .. ..
. . . . .
xB(ℓ) (B−1 A1 )ℓ ... uℓ ... 0 ... (B−1 An )ℓ
.. .. .. .. ..
. . . . .
xB(i) (B−1 A1 )i ... ui ... 1 ... (B−1 An )i
.. .. .. .. ..
. . . . .
xB(m) (B−1 A1 )m . . . um . . . 0 . . . (B−1 An )m

▶ Hence, adding a multiple of the pivot row to the zeroth row

of the tableau does not affect the zeroth row entry of that
column, which is left at zero.
▶ Thus for i ̸= ℓ we have cB̄(i) − p′ AB̄(i) = 0.
The zeroth row

▶ a) and b) imply that the vector p satisfies

cB̄(i) − p′ AB̄(i) = 0 i = 1, . . . , m.

▶ In matrix form we have

c′B̄ − p′ B̄ = 0 ⇐⇒ p′ = c′B̄ B̄−1 .

▶ Hence, with our update rule, the updated zeroth row of the
tableau is equal to

[0 | c′ ] − p′ [b | A] = [0 | c′ ] − c′B̄ B̄−1 [b | A],

as desired.
The full tableau implementation
We can now summarize the mechanics of the full tableau
implementation.

An iteration of the full tableau implementation

1. A typical iteration starts with the tableau associated with
a basis matrix B and the corresponding basic feasible
solution x.
2. Examine the reduced costs in the zeroth row of the
tableau.
▶ If they are all nonnegative, the current basic feasible
solution is optimal, and the algorithm terminates.
▶ Else, choose some j for which c̄j < 0.
3. Consider the vector u = B−1 Aj , which is the jth column
(the pivot column) of the tableau. If no component of u
is positive, the optimal cost is −∞, and the algorithm
terminates.
The full tableau implementation

An iteration of the full tableau implementation

4. For each i for which ui is positive, compute the ratio
xB(i)
.
ui
Let ℓ be the index of a row that corresponds to the
smallest ratio. The column AB(ℓ) exits the basis and the
column Aj enters the basis.
5. Add to each row of the tableau a constant multiple of the
ℓth row (the pivot row) so that uℓ (the pivot element)
becomes one and all other entries of the pivot column
become zero.
Example 3.5

minimize − 10x1 − 12x2 − 12x3

subject to x1 + 2x2 + 2x3 ≤ 20
2x1 + x2 + 2x3 ≤ 20
2x1 + 2x2 + x3 ≤ 20
x1 , x2 , x3 ≥ 0.
Example 3.5
After introducing slack variables, we obtain the standard form
problem
minimize − 10x1 − 12x2 − 12x3
subject to x1 + 2x2 + 2x3 + x4 = 20
2x1 + x2 + 2x3 + x5 = 20
2x1 + 2x2 + x3 + x6 = 20
x1 , x2 , x3 , x4 , x5 , x6 ≥ 0.

minimize − 10x1 − 12x2 − 12x3

subject to x1 + 2x2 + 2x3 ≤ 20
2x1 + x2 + 2x3 ≤ 20
2x1 + 2x2 + x3 ≤ 20
x1 , x2 , x3 ≥ 0.
Example 3.5
After introducing slack variables, we obtain the standard form
problem
minimize − 10x1 − 12x2 − 12x3
subject to x1 + 2x2 + 2x3 + x4 = 20
2x1 + x2 + 2x3 + x5 = 20
2x1 + 2x2 + x3 + x6 = 20
x1 , x2 , x3 , x4 , x5 , x6 ≥ 0.

▶ Note that x = (0, 0, 0, 20, 20, 20) is a basic feasible solution

and can be used to start the algorithm.
▶ Let accordingly, B(1) = 4, B(2) = 5, and B(3) = 6.
▶ The corresponding basis matrix is the identity matrix I.
▶ To obtain the zeroth row of the initial tableau, we note that
cB = 0 and, therefore, c′B xB = 0 and c̄′ = c′ − c′B B−1 A = c′ .
▶ Hence, we have the following initial tableau:
Example 3.5: Initial tableau
x1 x2 x3 x4 x5 x6
0 −10 −12 −12 0 0 0
x4 = 20 1 2 2 1 0 0
x5 = 20 2 1 2 0 1 0
x6 = 20 2 2 1 0 0 1
Example 3.5: Initial tableau
x1 x2 x3 x4 x5 x6
0 −10 −12 −12 0 0 0
x4 = 20 1 2 2 1 0 0
x5 = 20 2 1 2 0 1 0
x6 = 20 2 2 1 0 0 1

We note a few conventions in the format of the above tableau:

▶ The label xi on top of the ith column indicates the variable
associated with that column.
▶ The labels “xi =” to the left of the tableau tell us which are
the basic variables and in what order:
▶ xB(1) = x4 = 20,
▶ xB(2) = x5 = 20,
▶ xB(3) = x6 = 20.
Example 3.5: Initial tableau
x1 x2 x3 x4 x5 x6
0 −10 −12 −12 0 0 0
x4 = 20 1 2 2 1 0 0
x5 = 20 2 1 2 0 1 0
x6 = 20 2 2 1 0 0 1

We note a few conventions in the format of the above tableau:

▶ These labels are not necessary.

▶ We know that the column in the tableau associated with
the first basic variable must be the first unit vector.
▶ Once we observe that the column associated with the
variable x4 is the first unit vector, it follows that x4 is the
first basic variable.
Example 3.5: Initial tableau
x1 x2 x3 x4 x5 x6
0 −10 −12 −12 0 0 0
x4 = 20 1 2 2 1 0 0
x5 = 20 2 1 2 0 1 0
x6 = 20 2 2 1 0 0 1

We continue with our example.

▶ The reduced cost of x1 is negative and we let that variable
enter the basis.
▶ The pivot column is u = (1, 2, 2).
▶ We form the ratios xB(i) /ui , i = 1, 2, 3:
▶ xB(1) /u1 = 20/1 = 20,
▶ xB(2) /u2 = 20/2 = 10,
▶ xB(3) /u3 = 20/2 = 10.
▶ The smallest ratio corresponds to i = 2 and i = 3.
▶ We break this tie by choosing ℓ = 2.
Example 3.5: Initial tableau
x1 x2 x3 x4 x5 x6
0 −10 −12 −12 0 0 0
x4 = 20 1 2 2 1 0 0
x5 = 20 2 1 2 0 1 0
x6 = 20 2 2 1 0 0 1

▶ The second basic variable xB(2) , which is x5 , exits the basis.

This determines the pivot row and the pivot element.
▶ The new basis is given by B̄(1) = 4, B̄(2) = 1, and
B̄(3) = 6.
Example 3.5: Initial tableau
x1 x2 x3 x4 x5 x6
0 −10 −12 −12 0 0 0
x4 = 20 1 2 2 1 0 0
x5 = 20 2 1 2 0 1 0
x6 = 20 2 2 1 0 0 1

▶ We multiply the pivot row by 5 and add it to the zeroth

row.
▶ We multiply the pivot row by 1/2 and subtract it from the
first row.
▶ We subtract the pivot row from the third row.
▶ Finally, we divide the pivot row by 2.
▶ This leads us to the new tableau:
Example 3.5: First pivot
x1 x2 x3 x4 x5 x6
100 0 −7 −2 0 5 0
x4 = 10 0 1.5 1 1 −0.5 0
x1 = 10 1 0.5 1 0 0.5 0
x6 = 0 0 1 −1 0 −1 1

▶ The cost has been reduced to −100.

▶ The corresponding basic feasible solution is
x = (10, 0, 0, 10, 0, 0).
▶ Note that this is a degenerate basic feasible solution,
because the basic variable x6 is equal to zero.
Example 3.5: First pivot
x1 x2 x3 x4 x5 x6
100 0 −7 −2 0 5 0
x4 = 10 0 1.5 1 1 −0.5 0
x1 = 10 1 0.5 1 0 0.5 0
x6 = 0 0 1 −1 0 −1 1

▶ In terms of the original variables x1 , x2 , x3 , we have moved

to the degenerate solution D = (10, 0, 0).
Example 3.5: First pivot
x1 x2 x3 x4 x5 x6
100 0 −7 −2 0 5 0
x4 = 10 0 1.5 1 1 −0.5 0
x1 = 10 1 0.5 1 0 0.5 0
x6 = 0 0 1 −1 0 −1 1

▶ We have mentioned earlier that the rows of the tableau

(other than the zeroth row) amount to a representation of
the equality constraints B−1 Ax = B−1 b, which are
equivalent to the original constraints Ax = b.
▶ In our current example, the tableau indicates that the
equality constraints can be written in the equivalent form:

10 = 1.5x2 +x3 +x4 −0.5x5

10 = x1 +0.5x2 +x3 +0.5x5
0= x2 −x3 −x5 +x6 .
Example 3.5: First pivot
x1 x2 x3 x4 x5 x6
100 0 −7 −2 0 5 0
x4 = 10 0 1.5 1 1 −0.5 0
x1 = 10 1 0.5 1 0 0.5 0
x6 = 0 0 1 −1 0 −1 1

▶ We now return to the simplex method.

▶ With the current tableau, the variables x2 and x3 have
negative reduced costs.
▶ We choose x3 to be the one that enters the basis.
▶ The pivot column is u = (1, 1, −1).
▶ Since u3 < 0, we only form the ratios xB(i) /ui , for i = 1, 2:
▶ xB(1) /u1 = 10/1 = 10,
▶ xB(2) /u2 = 10/1 = 10.
▶ There is again a tie, which we break by letting ℓ = 1.
Example 3.5: First pivot
x1 x2 x3 x4 x5 x6
100 0 −7 −2 0 5 0
x4 = 10 0 1.5 1 1 −0.5 0
x1 = 10 1 0.5 1 0 0.5 0
x6 = 0 0 1 −1 0 −1 1

▶ The first basic variable, x4 , exits the basis.

This determines the pivot row and the pivot element.
▶ We multiply the pivot row by 2 and add it to the zeroth
row.
▶ We subtract the pivot row from the second row.
▶ Finally, we add the pivot row to the third row.
▶ We obtain the following new tableau:
Example 3.5: Second pivot
x1 x2 x3 x4 x5 x6
120 0 −4 0 2 4 0
x3 = 10 0 1.5 1 1 −0.5 0
x1 = 0 1 −1 0 −1 1 0
x6 = 10 0 2.5 0 1 −1.5 1

▶ The cost has been reduced to −120.

▶ The corresponding basic feasible solution is
x = (0, 0, 10, 0, 0, 10).
Example 3.5: Second pivot
x1 x2 x3 x4 x5 x6
120 0 −4 0 2 4 0
x3 = 10 0 1.5 1 1 −0.5 0
x1 = 0 1 −1 0 −1 1 0
x6 = 10 0 2.5 0 1 −1.5 1

▶ In terms of the original variables x1 , x2 , x3 , we have moved

to point B = (0, 0, 10).
Example 3.5: Second pivot
x1 x2 x3 x4 x5 x6
120 0 −4 0 2 4 0
x3 = 10 0 1.5 1 1 −0.5 0
x1 = 0 1 −1 0 −1 1 0
x6 = 10 0 2.5 0 1 −1.5 1

▶ At this point, x2 is the only variable with negative reduced

cost.
▶ We bring x2 into the basis.
▶ The pivot column is u = (1.5, −1, 2.5).
▶ Since u2 < 0, we only form the ratios xB(i) /ui , for i = 1, 3:
▶ xB(1) /u1 = 10/1.5 = 6.6̄,
▶ xB(3) /u3 = 10/2.5 = 4.
▶ We obtain ℓ = 3, and the third basic variable, x6 exits the
basis.
Example 3.5: Second pivot
x1 x2 x3 x4 x5 x6
120 0 −4 0 2 4 0
x3 = 10 0 1.5 1 1 −0.5 0
x1 = 0 1 −1 0 −1 1 0
x6 = 10 0 2.5 0 1 −1.5 1

▶ This determines the pivot row and the pivot element.

▶ We multiply the pivot row by 4/2.5 and add it to the
zeroth row.
▶ We multiply the pivot row by 1.5/2.5 and subtract it to the
first row.
▶ We multiply the pivot row by 1/2.5 and add it to the
second row.
▶ Finally, we divide the pivot row by 2.5.
▶ We obtain the following new tableau:
Example 3.5: Third pivot
x1 x2 x3 x4 x5 x6
136 0 0 0 3.6 1.6 1.6
x3 = 4 0 0 1 0.4 0.4 −0.6
x1 = 4 1 0 0 −0.6 0.4 0.4
x2 = 4 0 1 0 0.4 −0.6 0.4

▶ The cost has been reduced to −136.

▶ The corresponding basic feasible solution is
x = (4, 4, 4, 0, 0, 0).
Example 3.5: Third pivot
x1 x2 x3 x4 x5 x6
136 0 0 0 3.6 1.6 1.6
x3 = 4 0 0 1 0.4 0.4 −0.6
x1 = 4 1 0 0 −0.6 0.4 0.4
x2 = 4 0 1 0 0.4 −0.6 0.4

▶ In terms of the original variables x1 , x2 , x3 , we have moved

to point E = (4, 4, 4).
Example 3.5: Third pivot
x1 x2 x3 x4 x5 x6
136 0 0 0 3.6 1.6 1.6
x3 = 4 0 0 1 0.4 0.4 −0.6
x1 = 4 1 0 0 −0.6 0.4 0.4
x2 = 4 0 1 0 0.4 −0.6 0.4

▶ The optimality of this solution is confirmed by observing

that all reduced costs are nonnegative.
Example 3.5

▶ In this example, the simplex method took three changes of

basis to reach the optimal solution, and it traced the path
A − D − B − E.
▶ With different pivoting rules, a different path would have
been traced.
Example 3.5

▶ Question: Could the simplex method have solved the

problem by tracing the path A − D − E, which involves only
two edges, with only two iterations?
Example 3.5

▶ Question: Could the simplex method have solved the

problem by tracing the path A − D − E, which involves only
two edges, with only two iterations? The answer is no.
▶ The initial and final bases differ in three columns, and
therefore at least three basis changes are required.
▶ In particular, if the method were to trace the path
A − D − E, there would be a degenerate change of basis at
point D (with no edge being traversed), which would again
bring the total to three.
The full tableau implementation: running time

What is the total computational effort per iteration?

▶ The full tableau method requires a constant (and small)
number of arithmetic operations for updating each entry of
the tableau.
▶ Thus, the amount of computation per iteration is proportional
to the size of the tableau, which is

O(mn).

▶ Therefore, the full tableau method is as efficient as the revised

simplex method.
Comparison of the full tableau and the revised
simplex methods
Comparison of the full tableau and the revised simplex
▶ Consider a LP problem in standard form

minimize c′ x
subject to Ax = b
x ≥ 0.

▶ Let us pretend that the problem is changed to

minimize c′ x + 0′ y
subject to Ax + Iy = b
x, y ≥ 0.
Comparison of the full tableau and the revised simplex
▶ Consider a LP problem in standard form

minimize c′ x
subject to Ax = b
x ≥ 0.

▶ Let us pretend that the problem is changed to

minimize c′ x + 0′ y
subject to Ax + Iy = b
x, y ≥ 0.

▶ We implement the simplex method on this new problem,

except that we never allow any of the components of the
vector y to become basic.
▶ Then, we always have y = 0, and the simplex method
performs basis changes as if the vector y were entirely absent.
Comparison of the full tableau and the revised simplex
▶ The equality constraints of our new standard form problem
are Ax + Iy = b, thus the new constraint matrix is

[A | I].

▶ The vector of reduced costs in the augmented problem is

[c′ | 0′ ] − c′B B−1 [A | I] = [c̄′ | −c′B B−1 ].

▶ Thus, the simplex tableau for the augmented problem is

−c′B B−1 b c̄′ −c′B B−1

B b−1 −1
B A B−1

▶ If we follow the mechanics of the full tableau implementation

on the above tableau, the inverse basis matrix B−1 is updated
at each iteration.
Comparison of the full tableau and the revised simplex

−c′B B−1 b c̄′ −c′B B−1

B−1 b B−1 A B−1

▶ The revised simplex method is essentially the full tableau

method applied to the above augmented problem, except
that the part of the tableau containing c̄′ and B−1 A is
never formed explicitly.
▶ If the revised simplex method also updates the zeroth row
entries that lie on top of B−1 (by the usual elementary
operations), the simplex multipliers p′ = c′B B−1 become
available, thus eliminating the need for computing
p′ = c′B B−1 at each iteration.
Comparison of the full tableau and the revised simplex

−c′B B−1 b c̄′ −c′B B−1

B−1 b B−1 A B−1

▶ We can apply the revised simplex method with a pivoting

rule that evaluates one reduced cost at a time, until a
negative reduced cost is found.
▶ Once the entering variable xj is chosen, the pivot column
B−1 Aj is computed on the fly.
Comparison of the full tableau and the revised simplex

−c′B B−1 b c̄′ −c′B B−1

B−1 b B−1 A B−1

We now discuss the relative merits of the two methods.

▶ The full tableau method updates all the tableau at each
iteration.
▶ So the computational requirements per iteration are

O(mn).
Comparison of the full tableau and the revised simplex

−c′B B−1 b c̄′ −c′B B−1

B−1 b B−1 A B−1

▶ The revised simplex method updates B−1 and p′ = c′B B−1 .

[O(m2 ) operations]
▶ In addition, the reduced cost of each variable xj is
computed as p′ Aj , requiring O(m) operations. In the worst
case, the reduced cost of every variable is computed.
[O(mn) operations]
▶ Once the entering variable xj is chosen, the pivot column
B−1 Aj is computed on the fly as a matrix-vector product.
[O(m2 ) operations]
▶ Since m ≤ n, the worst-case computational effort per
iteration is
O(mn + m2 ) = O(mn).
Comparison of the full tableau and the revised simplex

−c′B B−1 b c̄′ −c′B B−1

B−1 b B−1 A B−1

▶ On the other hand, a typical iteration of the revised simplex

method might require a lot less work.
▶ In the best case, if the first reduced cost computed is
negative, and the corresponding variable is chosen to enter
the basis, the total computational effort is only

O(m2 ).

▶ The conclusion is that the revised simplex method cannot

be slower than the full tableau method, and could be much
faster during most iterations.
Comparison of the full tableau and the revised simplex

−c′B B−1 b c̄′ −c′B B−1

B−1 b B−1 A B−1

▶ Another important element in favor of the revised simplex

method is that memory requirements are reduced from
O(mn) to O(m2 ). Why?
▶ As n is often much larger than m, this effect can be quite
significant.
Comparison of the full tableau and the revised simplex

−c′B B−1 b c̄′ −c′B B−1

B−1 b B−1 A B−1

▶ It could be counterargued that the memory requirements of

the revised simplex method are also O(mn) because of the
need to store the matrix A.
▶ However, in most large scale problems that arise in
applications, the matrix A is very sparse (has many zero
entries) and can be stored compactly.
▶ The sparsity of A does not usually help in the storage of the
full simplex tableau because even if A and B are sparse,
B−1 A is not sparse, in general.
Comparison of the full tableau and the revised simplex

We summarize this discussion in the following table:

Full tableau Revised simplex

Worst-case time O(mn) O(mn)
Worst-case memory O(mn) O(mn)
Best-case time O(mn) O(m2 )
Best-case memory O(mn) O(m2 )