26 LP Basics II

3.3 Sensitivity Analysis

Analyse the stability of an optimal (primal or dual) solution

against the (plus and minus) changes of an coefficient in the LP.

There are two types of analyses that are computationally easy.

A. Stability of an dual optimal solution w.r.t. RHS changes
Fix one constraint, and consider the modified LP where the RHS in the fixed constraint
is changed by a parameter α.

max 3x1 + 4x2 + 2x3

subject to 2x1 ≤ 4+α
x1 + 2x3 ≤ 8
3x2 + x3 ≤ 6
x1 ≥ 0, x2 ≥ 0, x3 ≥ 0

Ranges of α for which the dual optimal solution (dual price)

4 1 4
(ȳ1 , ȳ2 , ȳ3 ) = ( , , )
3 3 3
stays optimal?

=⇒ -4 ≤ α ≤ 12
$ $
allowable decrease allowable increase

Remark 3.1 The standard technique to be discussed in Section 4.6 is not completely satis-
factory: it might return ranges that are not widest possible.

B. Stability of an optimal solution w.r.t. objective changes

Fix one coefficient of the objective function, and consider the modified LP where the
fixed objective coefficient is changed by a parameter β.

max 3x1 + (4 + β)x2 + 2x3

subject to 2x1 ≤ 4
x1 + 2x3 ≤ 8
3x2 + x3 ≤ 6
x1 ≥ 0, x2 ≥ 0, x3 ≥ 0
3.3 Sensitivity Analysis 27

Ranges of β for which the optimal solution

(x̄1 , x̄2 , x̄3 ) = (2, 1, 3)
stays optimal?

=⇒ -4 ≤ β ≤ 2
$ $
allowable decrease allowable increase

Remark 3.2 The same remark as Remark 3.1 applies to this case.

Sensitivity Analysis in LINDO

RANGES IN WHICH THE BASIS IS UNCHANGED

OBJ COEFFICIENT RANGES

VARIABLE CURRENT ALLOWABLE ALLOWABLE
COEF INCREASE DECREASE
X1 3.000000 INFINITY 2.666667
X2 4.000000 2.000000 4.000000 beta
X3 2.000000 5.333333 .666667

RIGHTHAND SIDE RANGES

ROW CURRENT ALLOWABLE ALLOWABLE
RHS INCREASE DECREASE
2 4.000000 12.000000 4.000000 alpha
3 8.000000 6.000000 6.000000
4 6.000000 INFINITY 3.000000

Sensitivity Analysis in CPLEX

OBJ Sensitivity Ranges

Variable Name Reduced Cost Down Current Up
x1 zero 0.3333 3.0000 +infinity
x2 zero zero 4.0000 6.0000
x3 zero 1.3333 2.0000 7.3333
---
Display what: rhs
Display RHS sensitivity for which constraint(s): c1-c3
RHS Sensitivity Ranges
Constraint Name Dual Price Down Current Up
c1 1.3333 zero 4.0000 16.0000
c2 0.3333 2.0000 8.0000 14.0000
c3 1.3333 3.0000 6.0000 +infinity
28 LP Basics II

Figure 3.2 exhibits geometric meaning of sensitivity analysis. There the supply of gamay
grape (the second constraint) is changed gradually from the current 8 tons to 1 ton. The
primal optimal solution is indicated by • which moves from the top point on the plane
of supply 8 to the bottom point on the supply 2 smoothly, and then turns right to the
point on the supply 1. This means the supply 2 which corresponds to the degrease of 6
tons (α = −6) is critical. In fact, when the supply becomes smaller than 2, the structure
of the feasible region becomes different. In particular the pinot grape constraint 2x1 ≤ 4
becomes nonbinding (since its supply is too much and cannot be fully used for any feasible
productions) and thus its price becomes zero. Therefore the sensitivity analysis returns the
allowable decrease as 6.

x3 Gamay supply

8
x1 + 2 x3 ≤ 8+ α
6

2 (α= -6)

x2

x1 2x1 ≤ 4

Figure 3.2: Sensitivity of the dual price with respect to changes in Gamay supply

One can easily imagine how the feasible region changes as α increases. The shape
smoothly changes as the supply of gamay increases until the constraint hyperplane itself
disappears from the feasible region to form a sharp pyramid. This critical α (allowable in-
crease) is 6 (as shown in the ouput of LINDO/CPLEX), and the dual price of gamay becomes
0 for gamay supply larger than 14 (= 8 + 6).
Chapter 4

LP Algorithms

The main goal of Chapter 4 is to present finite algorithms for solving a general LP. We
present two algorithms, the criss-cross method and the simplex method. The strong duality
theorem, Theorem 2.2, and two other theorems, Theorem 2.4 and Theorem 2.5, presented
earlier, are direct consequences of their finiteness.
Both algorithms rely on an elementary operation on matrices, called a pivot operation.
They both maintain a pair of primal solution x and dual solution y which satisfy the comple-
mentary slackness conditions in Theorem 2.3. They generate a new solution pair (x, y) using
a pivot operation until both solutions become feasible or a certificate for infeasibility of the
primal or the dual problem. The simplex method requires an initial primal feasible solution
and preserves the primal feasibility of x throughout, whereas the criss-cross method does
not require a feasible solution and may not preserve the feasibility even after it attained.
We shall not present interior-point algorithms in this chapter. These algorithms often
work for the convex nonlinear program, a larger class of optimization problems, and thus will
be given later in Chapter 10. One variation of such algorithms carry a dual pair of feasible
solutions (x, y), and generate a sequence of new feasible pairs until the complementary
slackness condition is “sufficiently” satisfied so that a simple modification of the solutions
leads to an optimal solution pair.
To make a clean presentation of the pivot algorithms, we introduce an extended notation
for matrices in which the rows and columns of matrices are indexed by finite sets, rather
than by consecutive integer sets. The conventional notation imposes annoying unnecessary
structures, such as consecutive indices 1, 2, . . . , n for both rows and columns, to mathematical
models to be represented by matrices. We believe that the new notation is natural and easy
to manipulate for those who studied the basic linear algebra with usual matrix notations.
The immediate benefit of the new notation is simple and precise descriptions of algo-
rithms. They are easily readable by humans and almost readable by computers at the same
time. This also makes implementations in high level languages straightforward.
30 LP Algorithms

4.1 Matrix Notations

For finite sets M and N, an M ×N matrix is a family of doubly indexed numbers or variables

A = (aij : i ∈ M, j ∈ N)

where each member of M is called a row index, each member of N is called a column index,
and each aij is called a (i, j)-component or (i, j)-entry.

For R ⊆ M and S ⊆ N, the R × S matrix (ars : r ∈ R, s ∈ S) is called a submatrix of

A, and will be denoted by ARS . We use simplified notations like, AR for ARN , A·S for AM S ,
Ai for A{i} , and A·j for A·{j}. For each i, j, Ai = (aij : j ∈ N) is called the i-row and the
matrix A·j = (aij : i ∈ M) is called the j-column. Again, more visually,

A= i Ai = A·j (i ∈ M, j ∈ N )

A= R AR = A·S (R ⊆ M, S ⊆ N )

M × N matrices might be simply called matrices if the row and column index sets are
clear from the context or irrelevant. Real, rational, integer matrices are matrices whose
components take real, rational, integer values, respectively.

For positive integer m, n, an m × n matrix is an M × N matrix where M and N are

some sets of cardinality m and n. Such m × n matrices will be used when there is no need
to specify the index sets. Unless otherwise specified, an m × n matrix should be considered
as M × N matrix with the usual index sets M = {1, . . . , m} and N = {1, . . . , n}. We shall
use mixed forms also: m × J matrix and I × n matrix whose meaning should be clear.

The negative −A and the scalar multiple αA of a matrix A are defined in the obvious
manner. The transpose AT of an M × N matrix A is the N × M matrix B = (bij : i ∈ N, j ∈
M) with bij = aji . The sum A + B of M × N matrices A and B is the M × N matrix C
A × B of an M × N matrix A and a N × K matrix B is
with cij = aij + bij . The product !
the M × K matrix C with cik = j∈N aij × bjk .
4.2 LP in Dictionary Form 31

An identity matrix is an K × K matrix A with aij = 1 if i = j and aij = 0 otherwise.

The K × K identity matrix is denoted by I (K) . A matrix of all zeros is denoted by 0. Its
row and column index sets are usually clear from the context.

An M × N matrix is called a column M-vector if the column index set N is a singleton

and identified with the set {1}. Similarly, an M × N matrix is called a row N-vector if
the row index set M is a singleton and identified with the set {1}. We use simple terms
row vectors and column vectors if there is no ambiguity. Unless otherwise specified, a vector
means a column vector. Row and column vectors are denoted by lower-case letters whose
components are singly indexed: x = (xj : j ∈ N). In the extreme case when both of the
index sets M and N are identified with {1}, such a matrix is both a row and column vector,
and is called a scalar.

When we say the i-row vector of a matrix A, we mean the row vector obtained from the
i-row Ai of A by identifying the only row index i with 1. Similarly we use the term j-column
(K)
vector . The j-column vector of a K × K identity matrix I (K) is denoted by ej for j ∈ K
and called a unit vector.

The set of all M × N real matrices is denoted by RM ×N , and the set of all real column
N-vectors is denoted by RN .

4.2 LP in Dictionary Form

An LP in canonical form is
max cT x
(4.1) subject to Ax ≤ b
x ≥ 0,
where A, b and c are given matrix and vectors.
A concrete example we use here is again Chateau ETH Production Problem, Example 1.1:
max 3x1 + 4x2 + 2x3
subject to
2x1 ≤ 4
x1 + 2x3 ≤ 8
3x2 + x3 ≤ 6
x1 ≥ 0, x2 ≥ 0, x3 ≥ 0.

To solve such an LP, it is convenient to convert all inequalities to equalities with extra
nonnegative variables, called slack variables. For example, the constraints can be converted
to an equivalent system by adding new variables x4 , x5 and x6 :
2x1 + x4 = 4
x1 + 2x3 + x5 = 8
3x2 + x3 + x6 = 6
x1 ≥ 0, x2 ≥ 0 x3 ≥ 0 x4 ≥ 0 x5 ≥ 0 x6 ≥ 0.
32 LP Algorithms

The indices 4, 5, 6 of slack variables represents the original inequality constraints (Pinot,
Gamay and Chasselas constraints, respectively) and the value of each slack variable is actu-
ally a slack or surplus of the supply, i.e. if x4 is positive at some feasible solution, the Pinot
grape supply 4 is not completely used and x4 is the surplus.
Also one can set the objective function cT x as a new variable xf as

−3x1 − 4x2 − 2x3 + xf = 0.

Therefore, the original LP is equivalent to

maximize xf = 0 + 3x1 + 4x2 + 2x3

subject to x4 = 4 − 2x1
x5 = 8 − x1 − 2x3
x6 = 6 − 3x2 − x3
x1 , x2 , x3 , x4 , x5 , x6 ≥0

Finally we use a new variable xg to make the equations homogeneous:

maximize xf = 0 · xg + 3x1 + 4x2 + 2x3

subject to x4 = 4 · xg − 2x1
x5 = 8 · xg − x1 − 2x3
x6 = 6 · xg − 3x2 − x3
xg = 1
x1 , x2 , x3 , x4 , x5 , x6 ≥0

The homogeneous equality system is then written as

xB = DxN

where D is the B × N matrix with B = {f, 4, 5, 6} and N = {g, 1, 2, 3}:

 
0 3 4 2
4 −2 0 0
D= 8 −1 0 −2 .


6 0 −3 −1

It is important to note that 0 in the first row and in the first column is not denoted by d11 but
df g , and −2 in the second row and in the second column is not denoted by d22 but d41 . This
non-conventional notation represents how the original variables are related to each other,
while the conventional notation completely ignores this correspondence and introduces some
arbitrary relations between the coincidently consecutively ordered rows and columns.
In general, if an LP is given in canonical form, we can transform it to

maximize xf
subject to
xf = 0 · xg + cT xN0
xB0 = b · xg − AxN0
xg = 1
xj ≥ 0 ∀j ∈ B0 ∪ N0 .
4.2 LP in Dictionary Form 33

where a new vector x is indexed by E = B0 ∪ N0 ∪ {f, g} and the original vector x is replaced
by xN0 .
By setting
( )
0 cT
(4.2) D= .
b −A

the canonical LP is equivalent to an LP in dictionary form:

maximize xf
subject to
(4.3) xB = DxN
xg = 1
xj ≥ 0 ∀j ∈ E \ {f, g}.

where E is a finite set partitioned into two disjoint subsets B and N, D is a given B × N
matrix, g ∈ N, f ∈ B. Here f is called the objective index, and g is the RHS (right hand
side) index. The set B is called a basis, N is a nonbasis and D is a dictionary. Note that
the dictionary D together with two elements f and g represent the LP.
The main goal of Chapter 4 is to present finite algorithms to solve a general LP in
dictionary form. We present two such algorithms, the criss-cross method and the simplex
method. The strong duality theorem is a consequence of their finiteness.
We define few basic terminologies. A feasible solution is a vector x ∈ RE satisfying
all constraints. An optimal solution is a feasible solution maximizing xf among all feasible
solutions. An LP is said to be feasible if it admits a feasible solution, and infeasible otherwise.
An unbounded direction is a vector z ∈ RE satisfying zf > 0, zg = 0 and all constraints except
zg = 1.
One can easily see that if an LP is infeasible or there is an unbounded direction, then
it cannot have an optimal solution. The fundamental fact for linear programming (Theo-
rem 1.3) is that the converse is true: an LP has an optimal solution if and only if it
is feasible and it admits no unbounded direction. This is a direct corollary of the
duality theorem to be discussed below.
The dual problem of an LP in dictionary form is defined as the following LP in dictionary
form:
maximize yg
subject to
(4.4) yN = −D T yB
yf = 1
yj ≥ 0 ∀j ∈ E \ {f, g}.

Thus the basis and the nonbasis are interchanged, f and g are interchanged, and the dictio-
nary is replaced by its negative transpose.

Proposition 4.1 The definition (4.4) of the dual LP in dictionary form extends the defini-
tion (2.6) of the dual LP in canonical form.
34 LP Algorithms

Proof. Let D be the reduction matrix (4.2) of a canonical form LP to a dictionary form
LP. Since
( )
T 0 −bT
−D = .
−c AT

the dual LP of the primal LP given by D is

maximize yg
subject to
yg = 0 · yf − bT yB0
(4.5)
yN0 =− c · yf + AT yB0
yf = 1
yj ≥ 0 ∀j ∈ B0 ∪ N0 .

We can then substitute yf with 1 and remove yN0 to get an equivalent LP:

min bT yB0
(4.6) subject to AT yB0 ≥ c
yB0 ≥ 0.

This problem is exactly the dual of the LP in canonical form defined in (2.6) if we consider
the variables yB0 as y.

We remark two important things with the proof above. First, the dual objective function
yg in the dictionary form (4.5) represents −bT y and this explains why the dual is also
maximization in dictionary form. Secondly, the variable yB0 in the reduced dual LP (4.6) in
canonical form is indexed by the row index set B0 of A. Of course this is natural because
the dual variables correspond to the primal constraints associated with the rows of A. In
the concrete example above, B0 = {4, 5, 6} and thus

min 4y4 + 8y5 + 6y6

subject to 2y4 + y5 ≥ 3
(4.7) 3y6 ≥ 4
2y5 + y6 ≥ 2
y4 ≥ 0, y5 ≥ 0, y6 ≥ 0

This LP is equivalent to the dual LP in Example 2.1 whose variables are y1 , y2 , y3 rather
than y4 , y5 , y6 . The form of Example 2.1 is widely used and considered standard. Yet our
form (4.7) is more appropriate theoretically, since the indices 1, 2, 3 represent the columns
of A (red, white, rose wines), where as 4, 5, 6 represent the different objects (Pinot, Gamay,
Chasselas grapes), the rows of A, and they correspond to the primal variables x4 , x5 , x6 ,
One simple but important observation for a dual pair of LPs in dictionary format is that

(4.8) xT y = xTB yB + xTN yN = 0

4.2 LP in Dictionary Form 35

for any primal and dual feasible solutions x and y. In fact this orthogonality of primal
and dual vectors is true as long as they satisfy the dictionary equalities: xB = DxN and
yN = −D T yB .
For any fixed j ∈ N, by setting xj = 1, xN −j = 0, and xB = D·j for the primal, we
obtain a vector, denoted by x(B, j), satisfying the dictionary equality system. In particular,
the vector x(B, g), satisfying all the equalities including xg = 1, is called the (primal) basic
solution with respect to B. The remaining vectors x(B, j) for j ∈ N − g have zero g
component and are called basic directions.
Similarly, for any fixed i ∈ B, by setting y i = 1, y B−i = 0, and y N = −(Di )T for the
dual, we obtain a vector, denoted by y(B, i) satisfying all dual dictionary equality system.
The vector y(B, f ) is called the dual basic solution and vectors y(B, i) for i ∈ B − f are the
dual basic directions.

Basic Solutions and Basic Directions (Reminder)

The basic vector x(B, j) for j ∈ N is

the unique solution x to xB = DxN such that

xj = 1
xN −j = 0
xB = D·j .

x(B, g) : the basic solution w.r.t. B

x(B, j) : the basic direction w.r.t. B and j ∈ N − g

The dual basic vector y(B, i) for i ∈ B is

the unique solution y to yN = −D T yB such that

yi = 1
y B−i = 0
y N = −(Di )T .

y(B, f ) : the dual basic solution w.r.t. B

y(B, i) : the dual basic direction w.r.t. B and i ∈ B − f

We state the weak and the strong duality theorems for LPs in dictionary form:

Theorem 4.2 (LP Weak Duality) For any LP in dictionary form (4.3) and for any pri-
mal and dual feasible solutions x and y, xf + yg ≤ 0.

Proof. Let x and y be primal and dual feasible solutions. Since xT y = 0 as remarked in
36 LP Algorithms

(4.8),

xf + yg = xf yf + xg yg ( ∵ xg = 1 and yf = 1)
*
= xT y − xj yj
j∈E\{f,g}
*
=− xj yj
j∈E\{f,g}

≤ 0. (∵ xj ≥ 0 and yj ≥ 0 for all j ∈ E \ {f, g})

One corollary of the weak duality is: if the primal LP is unbounded, then the dual LP is
infeasible. This is true because if the primal LP is unbounded and y is dual feasible, there
is a primal feasible solution x with xf as large as one wants, for example 1 − yg , for which
xf + yg ≤ 0 fails. The other consequence is: if a primal feasible solution x and a dual feasible
solution y satisfy xf + yg = 0 then both of them are optimal. The strong duality theorem
below shows that one can always verify the optimality of a feasible solution by exhibiting a
dual feasible solution satisfying this equality.

Theorem 4.3 (LP Strong Duality) For any LP in dictionary form (4.3), the following
statements hold:

(a) If the primal and dual LPs are both feasible then both have optimal solutions. Further-
more their optimal values sum up to zero.

(b) If one of the primal or dual LPs is infeasible then neither of the LPs has an optimal
solution. Moreover, if the dual (primal) LP is infeasible, the primal (dual, respectively)
LP is either infeasible or unbounded.

We shall prove this theorem in Section 4.4.

The dictionary form LP (4.1) is ideal for presenting certain algorithms for solving an
LP. Such algorithms use pivot operations to replace the current basis B with a new basis
B − r + s for some r ∈ B and s ∈ N and to update the dictionary D so that the associated
linear equality system stays equivalent (and thus the LP stays equivalent). We keep pivot-
ing until the associated basic solution is optimal or we find certificates for the LP having
no optimal solution. Dictionary facilitates simple certificates for optimality, infeasibility,
unboundedness, etc.
We define four different types of dictionaries (and bases). A dictionary D (or the associ-
ated basis B) is called (primal) feasible if dig ≥ 0 for all i ∈ B − f . A dictionary D (or the
associated basis B) is called dual feasible if df j ≤ 0 for all j ∈ N − g. A dictionary D (or
the associated basis B) is called (primal) inconsistent if there exists r ∈ B − f such that
drg < 0 and drj ≤ 0 for all j ∈ N − g. A dictionary D (or the associated basis B) is called
dual inconsistent if there exists s ∈ N − g such that df s > 0 and dis ≥ 0 for all i ∈ B − f .
4.2 LP in Dictionary Form 37

g g
f f - ··· -
⊕
..
.
⊕
(a) feasible (c) dual feasible

g g ∃s
f f +
⊕
..
∃r − - ··· - .
⊕
(b) inconsistent (d) dual inconsistent
Figure 4.1: Four types of dictionaries for LP

The four types of dictionaries are illustrated in Figure 4.1. Note that the symbols +, −, ⊕
and - denote positive, negative, nonnegative and nonpositive entries.

Proposition 4.4 For any LP in dictionary form, the following statements hold.

(a) If the dictionary is feasible then the associated basic solution is feasible.
(b) If the dictionary is dual feasible then the associated dual solution is feasible.
(c) If the dictionary is both primal and dual feasible then the associated basic solution x and
the associated dual basic solution y are optimal, and furthermore xf + y g = 0.

Proof.

(a) If the dictionary is feasible, the basic solution x(B, g) satisfies all the nonnegativity
conditions and thus is feasible.
(b) If the dictionary is dual feasible, the dual basic solution y(B, f ) satisfies all the nonneg-
ativity conditions and thus is dual feasible.
(c) Assume the dictionary is both primal and dual feasible. Since x(B, g)f = df g and
y(B, f )g = −df g , they add up to zero. By the Weak Duality, Theorem 4.2, they are
both optimal solutions.

One might profit from a direct way to see the optimality of x(B, g) when the dictionary
is both feasible and dual feasible (Statement (c) above). Observe the equality constraint:
*
xf = df g xg + df j xj
j∈N −g
38 LP Algorithms

that must be satisfied by any primal feasible solution. The basic solution is feasible and
attains the objective value df g . By the dual feasibility df j ≤ 0 for all j ∈ N − g, xf ≤ df g
for any feasible solution x, since any feasible x must satisfy xj ≥ 0 for any j ∈ N − g.
Because of the statement (c) above, a dictionary D (or the associated basis B) is called
optimal if it is both primal and dual feasible, see Figure 4.2.

g
f -···-
⊕
..
.
⊕
(e) optimal

Figure 4.2: Optimal dictionary

Proposition 4.5 For any LP in dictionary form, the following statements hold.

(a) If the dictionary is inconsistent then the LP is infeasible and the dual LP has an un-
bounded direction.

(b) If the dictionary is dual inconsistent then the dual LP is infeasible and the primal LP
has an unbounded direction.

Proof.

(a) Suppose the dictionary is inconsistent. Observe the row r indicating the inconsistency:
*
xr = drg xg + drj xj
j∈N −g

that must be satisfied by any primal feasible solution. Since drg < 0 and drj ≤ 0 for
all j ∈ N − g, xr < 0 for any x satisfying this equality, xg = 1 and xj ≥ 0 for all
j ∈ N − g. Therefore the LP has no feasible solution.
Now let w be the dual basic direction y(B, r). Then w satisfies all dual constraints
except for wf = 1, and we have wg > 0 and wf = 0. Thus w is a dual unbounded
direction.

(b) The proof is similar. Observe the corresponding dual constraint.

4.3 Pivot Operation 39

4.3 Pivot Operation

In this section, we present two finite algorithms for LP. Both algorithms transform an LP in
dictionary form to an equivalent LP in dictionary form with a modified dictionary using an
elementary matrix operation. This operation is called a pivot operation.
Suppose we have an LP in dictionary form (4.3), and consider the dictionary equality:

(4.9) xB = DxN

where D is a B × N matrix, and (B, N) is a partition of E. Denote the set of solutions to

the system by

(4.10) V (D) := {x ∈ RE : xB = DxN }.

Now suppose that r ∈ B, s ∈ N, and the (r, s)-entry drs of D is nonzero. Then one can
transform the equality system (4.9) to an equivalent system with the new basis B $ = B −r+s
and the new nonbasis N $ = N − s + r by simple substitution:

• First solve the r-th equation in (4.9):

*
xr = drj xj
j∈N

in terms of xs to obtain:
* drj 1
(4.11) xs = − xj + xr .
j∈N −s
drs drs

• Then eliminate xs using this equation in every other equations in (4.9) to get:
*
xi = dij xj
j∈N
*
= dij xj + dis xs
j∈N −s
+ ,
(4.12) * *drj 1
= dij xj + dis − xj + xr
j∈N −s j∈N −s
drs drs
* - dis · drj
.
dis
= dij − xj + xr ( for all i ∈ B − r).
j∈N −s
d rs d rs

• To summarize, by defining the B $ × N $ matrix D $ = [d$ij ] by

 1

 drs
if i=s and j =r

− drj if i=s and j .= r
(4.13) d$ij = disdrs (i ∈ B $ and j ∈ N $ ),

 if i .= s and j =r
 drs
 dis ·drj
dij − drs
if i .= s and j .= r
40 LP Algorithms

we obtain a new system of equations that is equivalent to the original system (4.9):

(4.14) xB! = D $ xN ! .

We say that the matrix D $ is obtained by a pivot operation in D on the position (r, s).
See Figure 4.3 for a schematic description.

j s
.. ..
. .
i ··· dij ··· dis ···
.. ..
D= . . (i ∈ B − r, j ∈ N − s)
r ··· drj ··· drs ···
.. ..
. .

⇓ Pivot on (r, s)

j r
.. ..
. .
dis · drj dis
i ··· dij − ··· ···
drs drs
.. ..
D$ = . . (i ∈ B − r, j ∈ N − s)
drj 1
s ··· − ··· ···
drs drs
.. ..
. .

Figure 4.3: Pivot Operation on (r, s)

We state the most important facts on pivot operations.

Proposition 4.6 Suppose D $ is obtained by a pivot operation in D on the position (r, s).
Then the following statements hold:

(a) (Reversibility) The original dictionary D is obtained from D $ by a pivot operation on

(s, r).

(b) (Equivalence) The associated equation systems are equivalent, that is, the set of solutions
are identical V (D) = V (D $ ).

(c) (Dual Equivalence) The associated dual equation systems are equivalent, that is, V (−D T ) =
V (−D $ T ).

Proof.

(a) This follows from the definition with straightforward calculation.

4.4 Pivot Algorithms and Constructive Proofs 41

(b) By the construction, every solution of the original system for D satisfies the new system,
i.e., V (D) ⊆ V (D $ ). The converse relation follows from (a).

(c) Let U = −D T . Since usr = −drs .= 0, we can perform a pivot operation in U on

(s, r) to obtain a new matrix U $ . It is easy to see that U $ = −D $ T . By (b), we have
V (U) = V (U $ ) and this proves (c).

Finally, we note that the pivot operation can be interpreted in terms of matrix multipli-
cations. The dictionary system (4.9)

xB = DxN

can be written as
( )
(B) xB
(4.15) [I − D] = 0.
xN

By setting A = [I (B) − D] (B × E matrix ), the system is

A x = 0.

The large matrix A is known as the tableau associated with the dictionary system. One can
represent the dictionary system by tableaus or dictionaries1 .
Now suppose ars (= −drs ) .= 0. Setting B $ = B − r + s and N $ = N + r − s, it can be
shown (how?) that the matrix A·B! has the left inverse, i.e. a unique B $ × B matrix T such
that T A·B! = I (B ) . We denote this matrix by A−1
!
·B ! . By multiplying this matrix from the
left of (4.15), we obtain a new equivalent system:

(4.16) A−1
·B ! A x = 0.

Since
!
(4.17) A−1
·B ! A = [I
(B )
A−1
·B ! A·N ! ]

the new dictionary D $ obtained by a pivot on (r, s) can be considered as

(4.18) D $ = −A−1
·B ! A·N ! .

4.4 Pivot Algorithms and Constructive Proofs

We shall call an LP dictionary D terminal if it is optimal, inconsistent or dual inconsistent.

In this section, we present finite algorithms that transforms any dictionary to a terminal
dictionary.
1
Our preference in this note is dictionary since it is more compact and it represents LP duality nicely.
42 LP Algorithms

The first one, called the (least-index) criss-cross method is one of the simplest pivot
algorithms. In fact it is almost trivial to code this algorithm. However it is in general quite
inefficient for large-scale problems. The second pivot method, called the simplex method , is
extremely practical but its description requires more space and care. The simplex method
has been implemented in many commercial codes and used to solve very large real-world
problems.
From the theoretical point of view, there is no polynomial-time pivot algorithm known
for linear programming. This means the number of pivot operations needed to solve an
LP of input size L might grow exponentially in L for both algorithms we present here.
Presently the known polynomial-time algorithms employ methods developed for non-linear
programming, such as ellipsoid methods and interior-point methods.

4.4.1 The Criss-Cross Method and a Proof of Strong Duality

The criss-cross method uses two types of pivot operations, both of which are quite natural.
For a B × N dictionary D with drs .= 0, a pivot on (r, s) is said to be admissible if one of
the following two conditions is satisfied (see Figure 4.4):

(I) drg < 0 and drs > 0;

(II) df s > 0 and drs < 0.

In the first case (I), the current dictionary is not feasible because drg < 0, but after the pivot
the new entry (d$ sg ) will become positive. For the second case (II), the dictionary is not
dual feasible because df s > 0, but after the pivot the corresponding entry will have a correct
(negative) sign. Furthermore, if a dictionary is not terminal, there is always an admissible
pivot.

g s g s
f f +

r − + r −

Type I Type II
Figure 4.4: Admissible Pivots

The (least-index) criss-cross method given below starts with any B × N dictionary D
with two specified elements f ∈ B and g ∈ N, and apply admissible pivots as long as the
dictionary is not terminal. Note E = B ∪ N denotes the set of all variable indices.
4.4 Pivot Algorithms and Constructive Proofs 43

procedure CrissCross(D,g,f );
begin
fix a linear order on E \ {f, g};
status:=”unknown”;
while status=”unknown” do
k := min({i ∈ B − f : dig < 0} ∪ {j ∈ N − g : df j > 0});
if no such k exists then
status:=”optimal”;
else
if k ∈ B then
r := k; s := min{j ∈ N − g : drj > 0};
if no such s exists then
status=”inconsistent”;
endif
else /* k ∈ N */
s := k; r := min{i ∈ B − f : dis < 0};
if no such r exists then
status:=”dual inconsistent”;
endif;
endif;
if status = ”unknown” then
make a pivot on (r, s);
replace D, B, N with the new D $ , B $ , N $ ;
endif;
endif;
endwhile;
output (status, D); /* D is terminal */
end.

Theorem 4.7 (Finiteness) The criss-cross method terminates after a finite number of
pivot operations.

Before proving this theorem, first observe the consequence.

Corollary 4.8 (Strong Duality Theorem in Dictionary Form) The dictionary of any
LP in dictionary form can be transformed by a finite sequence of pivot operations to a dic-
tionary that is optimal, inconsistent or dual inconsistent.

This corollary implies the strong duality theorem, Theorem 4.3. First of all, if the LP
admits an optimal solution, then both the primal and the dual LPs have optimal solutions
whose objective values sum up to zero by Proposition 4.4. If the LP is infeasible, then by
the corollary the LP admits an inconsistent or dual inconsistent dictionary. In either case,
the dual problem is either infeasible or unbounded by Proposition 4.5.
44 LP Algorithms

Directly or indirectly most (if not all) finiteness proofs of the least-index criss-cross
method rely on some fundamental proposition on “almost terminal” dictionaries. First we
present this proposition and a simple proof.
A dictionary is called almost terminal (with respect to a constraint index k ∈ E \ {f, g})
if it is not terminal but by discarding a single row or column (indexed by k) it becomes
terminal. Observe that we have four structurally different almost terminal dictionaries, see
Figure 4.5.

g k g
f - ··· - + f - ··· -
⊕ ⊕
.. ..
. .
· ⊕
⊕ k −
Type (A) Type (A*)

g k g ∃s
f f +
⊕
..
.
∃r − - ··· - + ⊕
k −
Type (B) Type (B*)
Figure 4.5: Four types of almost terminal dictionaries

Lemma 4.9 Consider an LP in dictionary form and fix an index k ∈ E \ {f, g}. Then the
LP admits (by pivot operations) at most one type of the four almost terminal dictionaries
A, B, A* and B* with respect to k.

Proof. We first transform a given LP to another LP (denoted by LP’) by substitution

of xk by its negative −xk . The four types of dictionaries A, B, A* and B* for the original
LP can be translated to the modified types a, b, a* and b* in Figure 4.6 for LP’, since the
substition affects as the reversal of signs in the row or the column associated with k.
We will show that no two different types from a, b, a* and b* can coexist for LP’, which
will then imply the lemma. There are six pairs of cases to consider. We denote by (a a*)
to mean the case that both types a and a* exist simultaneously, etc.
First of all, the types a and types a* are both optimal and thus indicate both the primal
and the dual LP’s are feasible. On the other hand the type b is an inconstent dictionary
indicating the infeasibility of the primal and the type (b*) is a dual inconsistent dictionary
indicating the dual infeasibility. Therefore all cases except (a a*) and (b b*) are impossible.
Consider the case (b b*), and let D, D $ be dictionaries of type b and b*, respectively.
Now look at the dual pair of directions: z = x(B $ , s) and w = y(B, r). By the definitions,
4.4 Pivot Algorithms and Constructive Proofs 45

g k g
f - ··· - − f - ··· -
⊕ ⊕
.. ..
. .
· ⊕
⊕ k +
Type (a) Type (a*)

g k g ∃s
f f +
⊕
..
.
∃r − - ··· - − ⊕
k +
Type (b) Type (b*)
Figure 4.6: Reduced four types

zk > 0 and wk > 0, and their remaining components are all nonnegative. This means
z T w > 0, contradicting the orthogonality of the spaces V (D) and V (−D T ) = V (−(D $ )T ).
To prove that the remaining case (a a*) is impossible, assume we have a dictionary D of
type a and a dictionary D $ of type a*. Let x and x$ be the basic solutions associated with
B and B $ , respectively. As we remarked,

(*) both x and x$ are optimal for LP’.

Moreover, observing the equation represented by the f -row of D, since df k < 0 and df j ≤ 0
for all j ∈ N − k, every feasible solution to LP’ with a positive k-component has an objective
value strictly less than the optimal value df g . Thus no optimal solution to LP’ can have a
positive kth component. Since x$k > 0, we have a contradiction.

Now we prove the finiteness of the criss-cross method, Theorem 4.7.

Proof. (of Theorem 4.7) Since the number of bases is finite, it is enough to show that
each basis can be visited at most once by the algorithm. The proof is by contradiction.
Assume that a basis is visited twice. Then the algorithm generates a cycle, i.e. starting from
this basis, after a certain number of steps the same basis is reached again. Without loss
of generality we may also assume that this cycling example is smallest possible in terms of
number of variables, and let E = {1, . . . , k} ∪ {f, g}. This implies each variable in E \ {f, g},
in particular the largest index k, must enter and leave a basis during the cycle. (If there is
any variable index j which stays in basis or nonbasis in the cycle, one can reduce the size of
the cycling example.)
There are two situations when k as a nonbasic element might enter a basis. The first
case A is when the current dictionary is almost optimal with respect to k, and the second
46 LP Algorithms

case B is when some basic variable, say r, is primal infeasible and drk determines the only
admissible pivot in row r. Similarly, there are only two cases in which k as a basic element
might leave a basis. The first case A* is when the current dictionary is almost optimal with
respect to k, and the second case B is when some nonbasic variable, say s, is dual infeasible
and dks determines the only admissible pivot in column s. Thus at least one case of the
four possible combinations (A A*), (A B*), (B A*) and (B B*) must occur. On the
other hand, the described cases A, B, A* and B* coincide with the four almost terminal
dictionaries of Figure 4.5. By Lemma 4.9, none of (A A*), (A B*), (B A*) and (B B*)
can occur, a contradiction.

4.4.2 Feasibility and Farkas’ Lemma

In this section, we prove the following feasibility theorem which is almost immediate from
the proof of the strong duality theorem in the previous section.

Theorem 4.10 (Feasibility Theorem in Dictionary Form) The dictionary of any LP

in dictionary form can be transformed by a finite sequence of pivot operations to a dictionary
that is either feasible or inconsistent.

Proof. The proof uses the criss-cross method to a modified LP. For a given LP with
dictionary D, we replace the objective row Df by the zero vector 0T . By applying the criss-
cross method to this problem, we will get a terminal dictionary, say D $ , by Corollary 4.8.
This dictionary cannot be dual inconsistent, since the dual problem is feasible. This means
we obtain either feasible or inconsistent dictionary. Clearly the same pivot sequence applied
to the original LP generates the same basis and a dictionary that is feasible or inconsistent.

Corollary 4.11 (Feasibility Theorem in Symmetric Form) For any matrix D ∈

B×N
R with B ∩ N = ∅, and for g ∈ N, exactly one of the following statements holds:

(a) ∃x ∈ RB∪N such that x ≥ 0, xg > 0 and xB = DxN .

(b) ∃y ∈ RB∪N such that y ≥ 0, yg > 0 and yN = −D T yB .

( T)
0
Proof. Consider the LP with (B + f ) × N dictionary D̄ = with the first row as
D
the objective row f . By Theorem 4.10, there is either a feasible dictionary or inconsistent
dictionary. If it is the first case, we have a feasible solution and thus (a). Otherwise, by
Proposition 4.5 (a), we have a dual unbounded direction y, i.e. yN = −D̄ T yB+f , yf = 0,
y ≥ 0 and yg > 0. This y without f component satisfies (b). It is clear that both (a) and
(b) cannot hold simultaneously.
4.4 Pivot Algorithms and Constructive Proofs 47

Corollary 4.12 (Farkas’ Lemma I) For any matrix A ∈ RB×N and any vector b ∈ RB ,
exactly one of the following statements holds:

(a) ∃x ∈ RN such that Ax ≤ b and x ≥ 0.

(b) ∃λ ∈ RB such that AT λ ≥ 0, λ ≥ 0 and bT λ < 0.

Proof. Let D = [b −A]. The claim follows from Corollary 4.11.

Corollary 4.13 (Farkas’ Lemma II) For any matrix A ∈ RB×N and any vector b ∈ RB ,
exactly one of the following statements holds:

(a) ∃x ∈ RN such that Ax = b and x ≥ 0.

(b) ∃λ ∈ RB such that AT λ ≥ 0 and bT λ < 0.

Proof. Exercise.
48 LP Algorithms

4.4.3 The Simplex Method

Unlike the criss-cross method which can start with any basis, the simplex method needs a
starting feasible basis. Because of this, it is often categorized as a two phase method . The
first phase, Phase I, is a procedure to find a feasible basis, and the second phase, Phase II,
starts with a feasible basis and finds a terminal basis.
First we present Phase II, since Phase I can be considered as an application of Phase II
to an artificially created problem for finding a feasible basis.

Phase II
The simplex method relies on one type of pivot operation. Let D be a B × N feasible
dictionary D. This means of course dig ≥ 0 for all i ∈ B −f . Consider the system xB = DxN
with xg fixed to 1 in detail:
*
(4.19) xf = df g + df j xj
j∈N −g
*
(4.20) xi = dig + dij xj (for all i ∈ B − f ).
j∈N −g

If the basis is optimal we have nothing to do: the current basic solution x is optimal.
Otherwise there is some nonbasic index s ∈ N − g with df s > 0. The current basic solution
fixes all nonbasic variables indexed by N −g to zero. Consider increasing xs by some positive
value $, while not modifying other nonbasic variables. The basic variables are determined
uniquely by the equations (4.19) and (4.20):

(4.21) x($)f = df g + df s $
(4.22) x($)i = dig + dis $ (for all i ∈ B − f ).

The objective value x($)f of this new solution increases by df s · $. Thus as long as the new
solution x($) stay feasible, this is a better feasible solution. And the feasibility is maintained
as long as x($)i ≥ 0 for all i ∈ B − f . The simplex method uses the largest $ that forces
some of the basic variables to become zero and one of them to leave the basis.
A pivot on (r, s) is called a simplex pivot if

(a) drs < 0;

(b) −drg /drs = min{−dig /dis : i ∈ B − f and dis < 0}.

A simple calculation shows that a simplex pivot preserves feasibility. This pivot exists as
long as the dictionary is feasible but not terminal. Here is the formal description of the
simplex method. The starting dictionary D must be feasible.
4.4 Pivot Algorithms and Constructive Proofs 49

procedure SimplexPhaseII(D,g,f );
begin
status:=”unknown”;
while status=”unknown” do
Let S = {j ∈ N − g : df j > 0};
if S = ∅ then
status:=”optimal”;
else
(L1) select any s ∈ S;
Let R := {i ∈ B − f : dis < 0};
if R = ∅ then
status:=”unbounded”;
else
(L2) make a simplex pivot on (r, s) for any r ∈ R;
replace D, B, N with the new D $ , B $ , N $ ;
endif;
endif;
endwhile;
output (status, D); /* D is terminal */
end.

Note that a simplex pivot may not modify the feasible solution itself. This happens
exactly when drg = 0. In general, a dictionary or its basis is called degenerate if drg = 0
for some r ∈ B − f . In fact, because of this, the simplex method may not terminate. See
Example 4.10.1 for an example for which the simplex method generates an (infinite) cycle
of pivots.
There are different ways to make the simplex method terminate in a finite number of
pivots. Two standard techniques are lexicographic rule and Bland’s rule. Both rules are
additional rules to restrict the selection of simplex pivots in (L1) and (L2) so as to avoid
cycling. Since Bland’s rule is simpler to explain, we present it here. We shall present the
lexicographic rule later.
The idea of Bland’s rule is to use a linear ordering on the index set and to break ties
with the minimum-index rule, just like the least-index criss-cross method selects pivots.

Definition 4.14 [Bland’s Rule or Smallest Index Rule]

(a) Fix any linear ordering on E \{f, g};

(b) In (L1), select the smallest s from S;

(c) In (L2), select the smallest r among all simplex pivot rows on column s.
50 LP Algorithms

Theorem 4.15 The simplex method (Phase II) with Bland’s rule is a finite algorithm.

Proof. Since the number of bases is finite, it is enough to show that each basis can be
visited at most once by the algorithm. The proof is by contradiction. Assume that a basis
is visited twice. Then the algorithm generates a cycle, i.e. starting from this basis, after a
certain number of steps the same basis is reached again. Without loss of generality we may
also assume that this cycling example is minimal and E = {1, . . . , k} ∪ {f, g}. This implies
each variable in E \ {f, g}, in particular k, must enter and leave a basis during the cycle.
When k as a nonbasic element enters a basis, the dictionary, say D, is almost optimal
except for df k has a wrong sign (i.e. positive). This is the situation, Type (A) in Figure 4.5.
On the other hand, when k as a basic element leaves a basis, the dictionary, say D $ , is almost
dual inconsistent except for d$ks has a wrong sign (i.e., negative) in some selected s column.
This is the situation, Type (B*) in Figure 4.5. By Lemma 4.9, these two types of dictionaries
cannot coexist, a contradiction.

See Section 4.10.5 for the cycling example solved by the simplex method with Bland’s rule.

Phase I
Now we describe the Phase I of the simplex algorithm. If the initial dictionary of a given
LP (4.3) is not feasible, one makes an auxiliary LP by adding an auxiliary variable and
setting an artificial objective xf ! in the original LP:

maximize xf !
subject to
xf ! = − xa
(4.23)
xB−f = DB−f xN + 1(B−f ) xa
xg = 1
xj ≥ 0 ∀j ∈ E ∪ {a} \ {f, g},

where 1(B−f ) is the vector of all 1’s indexed by B − f . Observe that the new objective
variable is −xa whose maximization forces xa to be zero if it is possible. It is easy to see
that the original LP is feasible if and only if the optimum value is 0. This LP is not yet
feasible, but a single pivot will make the dictionary primal feasible. More precisely, one can
select any pivot (r, a) satisfying

r ∈ B − f and drg = min{dig : i ∈ B − f }.

Phase I of the simplex method is essentially to run Phase II with this auxiliary LP. The
original objective function will be kept in the auxiliary dictionary throughout the Phase I.
Since the objective function of the auxiliary LP is bounded by zero, Phase I must terminate
in an optimal dictionary, say D̂. If the optimal value is negative, the original LP is infeasible.
Otherwise the optimal solution induces a feasible solution to the original LP. Moreover, one
can find a feasible basis easily. There are two cases. The first case is when the auxiliary
variable index a is nonbasic in the optimal basis. Then one can simply remove the a-column
and the f $ -row from D̂ to obtain a feasible dictionary of the original LP. The second case is
when the auxiliary variable index a is basic in D̂. In this case dˆag = 0 because the objective
4.5 Implementing Pivot Operations 51

value ( = −xa ) is zero at the basic solution. Now we make a pivot on (a, s) for any column
s with dˆas .= 0 and s ∈ N̂ − g. Such a pivot must exist, because otherwise xa would be
identically zero in the equality system of the auxiliary LP, that is impossible. Since dˆag = 0,
such a pivot will preserve the feasibility of the dictionary, and reduce this second case to the
first case.

4.5 Implementing Pivot Operations

In this section, we discuss how to interpret pivoting in terms of certain matrix operations.
The interpretation here is slightly different from one given in Section 4.3, and leads to an
implementation that is convenient for sensitivity analysis to be given in Section 4.6. There
are different ways for actual implementations but implementation details are beyond our
scope.
Here we assume that an LP is given in standard form:

max cT x
(4.24) subject to Ax = b
x ≥ 0,

where A is a given m × E matrix, b ∈ Rm and c ∈ RE . Since the row indices are not
important, the rows are {1, 2, . . . , m} The only difference with LP in canonical form is the
system Ax = b is an equality system instead of inequality Ax ≤ b. It is easy to reduce any
LP in canonical form to an equivalent LP in standard form.
We assume that A is row full rank, and we are given a basis of A, that is, a subset B ∈ E
such that |B| = m and the submatrix A·B has rank m. We call both the index set B and
the submatrix A·B basis of A. We denote by A−1·B the left inverse, the unique B × m matrix
T such that T A·B = I (B) .
Using the (left) inverse A−1
·B , one easily convert an LP in standard form to one in dictionary
form. First we solve Ax = b in terms of xB as

A−1 −1
·B (A·B xB + A·N xN ) = A·B b

which gives

(4.25) xB = A−1 −1
·B b − A·B A·N xN .

Now we substitute xB in the objective function by

cT x = cTB xB + cTN xN
3 4
(4.26) = cTB A−1 −1 T
·B b − A·B A·N xN + cN xN
3 T 4
= cTB A−1 T −1
·B b + cN − cB A·B A·N xN .
52 LP Algorithms

Let B = B + f , N = N + g with new indices f and g, E = B ∪ N, and set B × N matrix

D as
 T −1 
cB A·B b cTN − cTB A−1
·B A·N
(4.27) D= .
−1 −1
A·B b −A·B A·N

where the first row is indexed by f and the first column by g. Now we can rewrite an original
LP in standard form to an equivalent LP in dictionary form

maximize xf
subject to
(4.28) xB = DxN
xg = 1
xj ≥ 0 ∀j ∈ E \ {f, g}.

Hence we represent the dictionary associated with a basis B of the input matrix A in terms
of the original matrix data A, b and c and the basis inverse A−1
·B . In other words, a dictionary
can be computed as long as the input data and the current basis inverse A−1 ·B is at hand.

This leads to an implementation of any pivot algorithm based on basis-inverse updates.

We might call this technique revised pivot scheme:

(R1) Compute the inverse A−1

·B of an initial basis B.

(R2) Compute only the necessary part of the dictionary D using (4.27) for selecting a pivot
(r, s).

(R3) Once a pivot (r, s) is chosen, update B by B − r + s and compute the new inverse
A−1
·B from the old.

It should be remarked that the procedure (R3) is mathematically a simple matrix oper-
ation but its practical implementation is itself an important subject of matrix computation.
Practicality means many different things, memory/time efficiency, accuracy, robustness, etc.
We shall not discuss this issue here.
The simplex method with the revised pivot scheme is known as the revised simplex
method. Most (if not all) commercial codes use some variations of the revised scheme. There
are several reasons.

• Real-world LP problems have large and sparse input matrices. LP dictionaries, once
pivots are performed, tend to become dense very quickly. Basis inverses are often much
smaller than the input matrices and thus lots of memory can be saved.

• There are techniques, such as LU decomposition, of representing and updating basis

inverses, to control the accuracy of computation.

• Selection of a pivot usually depends only on a fraction of the current dictionary entries.
Thus updating the whole dictionary often wastes lots of time.
4.6 Computing Sensitivity 53

• Floating-point computation leads to accumulation of errors. It is extremely important

to keep the original data and to recompute the basis inverse or whatever by reliable
methods from time to time.

4.6 Computing Sensitivity

We consider LP in standard form in this section, because the necessary computation for the
analysis to be discussed comes quite naturally with the revised pivot scheme in the previous
section which is based on the standard form:

max cT x
(4.29) subject to Ax = b
x ≥ 0,

As we discussed in Section 4.5, a dictionary can be computed from the inverse A−1
·B of
the associated basis (of A) and the input matrix data as

 T −1 
cB A·B b cTN − cTB A−1
·B A·N
(4.30) D=  .
A−1
·B b −A−1
·B A·N

The sensitivity analysis is to evaluate the stability of a given optimal solution or a given
dual optimal solution. This question is usually modified to the same question for a given
optimal basis, because this is much easier to compute. Also the stability of an optimal basis
insures the stability of the associated (primal or dual) basic solutions.
The stability question for a given optimal basis (or dictionary) is to decide the widest
ranges of (plus and minus) changes of a given input data for which its optimality is preserved.
In principle any part of data is subject to change. However, certain analysis is much simpler
to perform than others.
First of all, we remind that the optimality of a dictionary D means

• (Primal Feasibility) A−1

·B b ≥ 0; and

• (Dual Feasibility) cTN − cTB A−1

·B A·N ≤ 0.

Now, if one wants to decide the range of ck for any fixed k ∈ E for which the current
dictionary stays optimal, it is just a matter of simplifying one variable inequality system
cTN − cTB A−1
·B A·N ≤ 0, where all cj ’s except for ck are constant.

Similarly, if one wants to decide the range of bk for any fixed k for which the current
dictionary stays optimal, it is a matter of simplifying one variable inequality system A−1
·B b ≥
0, where all bi ’s except for bk are constant.
54 LP Algorithms

These are two sensitivity analyses that are given by all standard LP software. It should
be noted that the ranges one obtains by these analyses are the widest possible for the
optimality of the given basis. It is possible that a wider range of ck (bk ) preserves the
optimality of the current primal (dual) solution2 . In this sense, a more accurate sensitivity
analysis is desirable, but to compute the widest range appears to be a much harder problem
and requires more sophisticated techniques.

4.7 Dualization of Pivot Algorithms

As we presented in Proposition 4.6, a pivot operation on a position (r, s) in a dictionary D

is the same thing as a pivot operation on (s, r) in the dual dictionary −D T , if we interpret
the result with its negative transpose.
Any pivot algorithm presented in terms of the primal dictionary, such as the simplex
method or the criss-cross method, can be applied to the dual LP without looking at the
dictionary of the dual LP. The dual simplex method is the simplex method applied to the
dual LP described with the primal dictionary D. The dual criss-cross method is exactly the
same method as the (primal) criss-cross method. Such a method is called self-dual.
Here is a description of the dual simplex method (Phase II), which needs an initial dual
feasible basis D.

procedure DualSimplexPhaseII(D,g,f );
begin
status:=”unknown”;
while status=”unknown” do
Let R := {i ∈ B − f : dig < 0};
if R = ∅ then
status:=”optimal”;
else
(L1) select any r ∈ R;
Let S := {j ∈ N − g : drj > 0};
if S = ∅ then
status:=”dual unbounded”;
else
(L2) make a dual simplex pivot on (r, s) for any s ∈ S;
replace D, B, N with the new D $ , B $ , N $ ;
endif;
endif;
endwhile;
output (status, D); /* D is terminal */
end.

Above, a pivot on (r, s) is called a dual simplex pivot if

2
This happens when the primal (dual) dictionary is degenerate.
4.8 Pivot Rules for the Simplex Method 55

(a) drs > 0;

(b) −df s /drs = min{−df j /drj : j ∈ N − g and drj > 0}.

One can easily show that by a dual simplex pivot the dual feasibility is preserved and
the objective value df g of the primal basic solution does not increase.
Which simplex method should one use, primal or dual? This question is extremely impor-
tant since they usually behave quite differently. We shall discuss this issue in Section 4.9.1.

4.8 Pivot Rules for the Simplex Method

The simplex method we presented in Section 4.4.3 admits multiple choices in selecting a
pivot at each iteration. This flexibility is critical for making the simplex method behave
favorably in one or more different criteria. For example, Bland’s rule (Definition 4.14) was
to make the simplex method finite. We shall present two simple rules that are useful for
making the simplex method practically efficient.

Definition 4.16 [Largest Coefficient Rule] In the step (L1) of SimplexPhaseII, select an
index s from S having the largest cost df s .

Definition 4.17 [Steepest Descent Rule] In the step (L1) of SimplexPhaseII, select an index
s from S having the largest δs , where

df s
δs = 5 .
Σi∈B−f d2is + 1

The quantity δs is essentially the amount of improvement in the objective5 function per
unit length of the associated basic direction x(B, s), since the denominator Σi∈B−f d2is + 1
is the length of the direction in the space of all variables except for xf and xg . The name
“steepest descent” comes from the classical algorithm to find the minimum of a function,
and employed here (and in standard textbooks) as it is for the maximization problem. Both
rules rely on the intuition that when the associated measure df s or δs is the largest, the
simplex pivot tends to make a relatively good improvement in the objective value.
It has been observed by extensive computational experiments that both rules usually
reduce the number of simplex pivots considerably, comparing with, for example, Bland’s
rule or the random rule:

Definition 4.18 [Random Rule (or Random-Edge Rule)] In the step (L1) of SimplexPha-
seII, select an index s from S randomly.

Furthermore, the steepest descent rule is considerably better than the largest coefficient rule
in terms of the number of pivots for solving many benchmark LPs. The main problem to
56 LP Algorithms

adopt the steepest descent rule is that the cost (time) to compute δs is quite expensive. Many
efficient implementation uses certain approximation of δs that is much easier to compute.
It is important to note that there is no known polynomial pivot rule. That is, a rule
that forces the simplex method to terminate in a number of pivots bounded above by a
polynomial function of m and d, the number of basic and nonbasic variables, respectively.
The positive resolution will be considered as a major breakthrough, because it would resolve
several other important open problems in geometry, optimization and combinatorics, as well.
There is a fascinating conjecture which had been open for over 35 years:

Conjecture 4.19 [Liebling 1975] The expected number of pivots performed by Random Rule
(Definition 4.18) is bounded above by a polynomial function of m and d.

In 2011, Friedmann-Hansen-Zwick [15] found counterexamples to this conjecture, for which

the Random Rule simplex method takes at least a superpolynomial number of pivots.
We also mention that there are important research developments on random pivot rules
whose expected behavior can be analyzed. In particular, there are pivot rules due to Kalai-
Kleitman [25] and to Sharir-Welzl [37] (1992) that attain sub-exponential bounds on the
expected number of pivots. Since many of the practical pivot rules such as the largest
coefficient rule were shown to require an exponential number of pivots in the worst case, any
bounds such as sub-exponential bounds that do not grow as fast as any exponential function
are interesting.
Finally, there is an indication that restricting to the simplex method might not be a
good strategy to find a polynomial-time pivot algorithm. While the criss-cross method is
exponential in general, the criss-cross method is conjectured to be an expected polynomial-
time method if the variables are randomly permuted at the beginning of the course of the
algorithm, see [13, 17].

Conjecture 4.20 [Fukuda 2008] The expected number of pivots performed by the criss-cross
method with random permutation of variables is bounded above by a polynomial function of
m and d.

For further readings on this subject, see Appendix A.

4.9 Geometry of Pivots 57

4.9 Geometry of Pivots

4.9.1 Geometric Observations of the Simplex Method

Until now, we have ignored the geometric aspects of linear programming. In this section, we
present some geometric interpretations of pivot operations.
To get some feeling about the geometry of LPs, look at Ch. ETH example,

max 3x1 + 4x2 + 2x3

subject to
E1: 2x1 ≤ 4
E2: x1 + 2x3 ≤ 8
E3: 3x2 + x3 ≤ 6
E4: x1 ≥ 0, x2 ≥ 0, x3 ≥ 0

and depict the sequence of feasible solutions generated by the simplex method, see Figure 4.7.
g 1 2 3
f 0 3 4 2
4 4 −2 0 0
5 8 −1 0 −2 x3
6 6 0 −3 −1
2
⇓ Pivot 1

g 1 2 5 3
f 8 2 4 −1
4 4 −2 0 0
1
3 4 −1/2 0 −1/2
6 2 1/2 −3 1/2

⇓ Pivot 2

g 1 6 5
x2
f 32/3 8/3 −4/3 −1/3
4 4 −2 0 0
3 4 −1/2 0 −1/2 x1
2 2/3 1/6 −1/3 1/6

⇓ Pivot 3 Figure 4.7: Feasible Region Ω and the

Trace of the Simplex Method
g 4 6 5
f 16 −4/3 −4/3 −1/3
1 2 −1/2 0 0
3 3 1/4 0 −1/2
2 1 −1/12 −1/3 1/6
58 LP Algorithms

The simplex method starts with the initial basic solution (0, 0, 0), and traces the path:
(0, 0, 0) −→ (0, 0, 4) −→ (0, 2/3, 4) −→ (2, 3, 1).
As we see in Figure 4.7, the simplex traces the boundary edges of the feasible region by
moving from a vertex (i.e., an extreme point in geometric language) to a neighboring vertex
which attains a better objective value.
We will prove formally in Section 4.9.3 that every basic feasible solution is in fact an
extreme point of the feasible region. (The definition of extreme point will be given later.)
From this geometric interpretation of the simplex method, one sees how the simplex
method will be affected by the combinatorial structure of the feasible region. For example,
the number of extreme points of a feasible region can be a very important measure for the
worst case bound for the number of simplex pivots. The combinatorial diameter of a convex
polyhedron determines how efficient an ideal simplex method can be. The combinatorial
diameter of Ω is the maximum of the lengths (in terms of number of edges) of shortest paths
between any pair of vertices. There is a famous conjecture3 by Hirsch that the diameter is
at most n − d for any d-dimensional convex polytope (i.e. bounded convex polyhedron)
given by a system of n inequalities. In the example above, d = 3 and n = 6, the conjecture
says one can connect any vertex to another by a path with at most 3 edges. The conjecture
is true for d = 3 and for various special cases, yet it is open in general.
Another interesting question is whether one should solve the primal LP or the dual LP,
for any given LP. If the size of matrix A in a given LP in canonical form is m × d, the feasible
region Ω in the primal LP is in Rd , and the dual Ω∗ in Rm . The number n of inequalities is
m + d for both problems. It has been observed (and proved for special cases) that the Hirsch
bound is a good measure for comparing the actual diameter. This means the Hirsch bound
m for the primal and d for the dual often indicate which polyhedron has smaller diameter.
In short, the primal region tends to have smaller diameter if its dimension d is larger than
the dual dimension, and larger diameter otherwise. Thus try to solve whichever problem
whose dimension is larger. If they are roughly equal, maybe it does not matter which one.
In Figure 4.8, we present small examples of Ch. ETH type where d is fixed to 3 and m
is chosen to be 3, 10, 50, 100. The values v(Ω) and diam(Ω) represent the number of vertices
and the diameter of a convex polyhedron Ω.
First of all, it is clear that the boundary of the primal feasible region becomes more
complex as m grows. The dual feasible region is in m-dimensional space and thus is hard
to draw, but the same remark is true for the dual regions. There are interesting contrasts
between the primal and the dual feasible regions: the primal region has much fewer vertices
than the dual region, but it has considerably larger diameter than the dual region. This trend
becomes stronger as m grows. In fact, when m is much larger than d, it is almost always
faster to solve the LP by the dual simplex method, the simplex method applied to the dual
problem, than by the primal simplex method. One can intuitively justify this observation
by the difference in diameters. In the dual LP, there is always a relatively short route to
the optimal vertex. Another way to understand this phenomenon is by looking at certain
special LP models and/or special (randomized) types of the simplex method.
3
The conjecture was disproved by Santos [34] in 2010. Yet, the Hirsch bound is only slightly (by a small
constant) exceeded by the construction and thus a slightly modified bound might be still valid.
4.9 Geometry of Pivots 59

Finally we see that the Hirsch bound is not valid for unbounded polyhedra. In the case
m = 10, the dual polyhedron has diameter 4, which is larger than the Hirsch bound 3. This
fact was proved first by Klee and Walkup in 1967.

d = 3, m = 3
v(Ω) = 8 diam(Ω) = 3
!
v(Ω∗ ) = 6 diam(Ω∗ ) = 3

d = 3, m = 10
v(Ω) = 22 diam(Ω) = 6
∧ ∨
v(Ω∗ ) = 92 diam(Ω∗ ) = 4

d = 3, m = 50
v(Ω) = 102 diam(Ω) = 12
∧ ∨
∗ ∗
v(Ω ) = 5983 diam(Ω ) = 4

d = 3, m = 100
v(Ω) = 202 diam(Ω) = 16
∧ ∨
∗ ∗
v(Ω ) = 44301 diam(Ω ) = ?

Figure 4.8: Primal Feasible Regions for

Increasing n in R3
60 LP Algorithms

4.9.2 Three Paradigms of LP Algorithms

We learned two pivot algorithms in this chapter, the simplex method and the criss-cross
method. There are many different variations of these methods as well. Later we present yet
another algorithm paradigm to linear programming, the interior-point algorithm.
The simplex method follows an edge path on the feasible region, the criss-cross method
follows a path in the whole space but stays on the lines generated by the constraint halfspaces
(1-skeleton of the arrangement of hyperplanes). The interior-point method follows a well-
defined nonlinear path in the whole space, see Figure 4.9. The criss-cross method admits
the simplest correct implementation. In terms of practicality the other two methods win
by far, at least with our present knowledge. And only the interior-point method enjoys
polynomial time complexity (for the moment). One might find these different characteristics
of LP algorithms rather annoying, but I consider them as clear indications for the richness
of LP Theory with fascinating mysteries.

3 Simplex Method

3
Criss-Cross Method 4

Interior-Point Method

Figure 4.9: Three Paradigms of LP Algorithms

4.9 Geometry of Pivots 61

4.9.3 Formal Discussions

Consider an LP in dictionary form,

maximize xf
subject to
(4.31) xB0 = DxN0
xg = 1
xj ≥ 0 ∀j ∈ E \ {f, g}.
We denote by Ω the set of all feasible solutions:
(4.32) Ω = {x ∈ RE : xB0 = DxN0 , xg = 1 and xj ≥ 0 ∀j ∈ E \ {f, g}}.
where we set E = B0 ∪ N0 . For each subset S of E, set
(4.33) Ω(S) = {xS ∈ RS : x ∈ Ω},
which is the orthogonal projection of Ω onto the subspace space RS . Two sets P ∈ RE and
Q ∈ RS are called affinely equivalent (denoted by P ∼ Q) if P and Q can be mapped to
each other by some affine bijections. Then one can verify the following:
(4.34) Ω ∼ Ω(N − g) for any nonbasis N of the dictionary system.
The less trivial part of constructing an affine map from Ω(N − g) can be done by using the
basic fact that xg = 1 and all basic variables xB are uniquely determined by the nonbasic
variables xN .
It is easy to see that the feasible region and its projections can be represented as
(4.35) Ω(S) = {x ∈ RS : Ax ≤ b}.
for some m × S matrix A and an S-vector b. Such a representation is useful to prove certain
properties of Ω(S).
In general, the set of solutions to a (finite) system of linear inequalities in RS is called
a convex polyhedron. The feasible region to a linear inequality system is exactly such a
set. For dimension d = |S| two or three, convex polyhedra can be visualized, but higher
dimensional polyhedra are not easy to “see”.4 In order to understand their structures, it is
convenient to make use of different representations of convex polyhedra. Here we rely only
on the definition itself, the inequality representation (or H-representation where H refers to
halfspaces).
For example, for Chateau ETH Problem, Example 1.1:
max 3x1 + 4x2 + 2x3
subject to
E1: 2x1 ≤ 4
E2: x1 + 2x3 ≤ 8
E3: 3x2 + x3 ≤ 6
E4: x1 ≥ 0, x2 ≥ 0, x3 ≥ 0
4
There are some techniques, such as Schlegal Diagrams and Gale Transformations, to visualize polyhedra
with special configurations.
62 LP Algorithms

the associated canonical system is

maximize xf = 0xg + 3x1 + 4x2 + 2x3

subject to x4 = 4xg − 2x1
x5 = 8xg − x1 − 2x3
x6 = 6xg − 3x2 − x3
xg = 1, x1 , x2 , x3 , x4 , x5 , x6 ≥0

and the feasible region is

Ω = {x = (x1 , x2 , . . . , x6 , xf , xg )T :
xf = 3x1 + 4x2 + 2x3
x4 = 4xg − 2x1
(4.36) x5 = 8xg − x1 − 2x3
x6 = 6xg − 3x2 − x3
xg = 1 and
x1 , x2 , x3 , x4 , x5 , x6 ≥ 0}.

Setting S1 = {1, 2, 3}, we have

Ω({1, 2, 3}) = {x ∈ RS1 :2x1 ≤ 4, x1 + 2x3 ≤ 8, 3x2 + x3 ≤ 6,

(4.37)
x1 ≥ 0, x2 ≥ 0, x3 ≥ 0}

Figure 4.10 shows this region which is affinely equivalent to Ω.

x3

x1 + 2 x3 ≤ 8

3 x2 + x3 ≤ 6

2 x1 ≤ 4

x2

x1

Figure 4.10: Feasible Region Ω({1, 2, 3})

4.9 Geometry of Pivots 63

x3
- x4 + 4x3 ≤ 12

3 x2 + x3 ≤ 6

x4 ≤ 4

x2
x4

Figure 4.11: Feasible Region Ω({4, 2, 3})

On the other hand, by setting S2 = {4, 2, 3}, we have

Ω({4, 2, 3}) = {x ∈ RS2 :x4 ≤ 4, −x4 + 4x3 ≤ 12, 3x2 + x3 ≤ 6,

(4.38)
x4 ≥ 0, x2 ≥ 0, x3 ≥ 0}

This can be verified if one performs a pivot on (4, 1) (exchanging x4 and x1 ) in the initial
dictionary to get

maximize xf = 6xg − 3/2x4 + 4x2 + 2x3

subject to x1 = 2xg − 1/2x4
x5 = 6xg + 1/2x4 − 2x3
x6 = 6xg − 3x2 − x3
xg = 1, x1 , x2 , x3 , x4 , x5 , x6 ≥ 0.

Figure 4.11 depicts the region Ω({4, 2, 3}). The regions Ω({1, 2, 3}) and Ω({4, 2, 3}) are
affinely equivalent. They simply use different coordinate systems to represent the same
object Ω. Note that the origins are the basic solutions of the associated bases. One might
already foresee that the basic solutions are “extreme” points of the feasible region.
Let us formally define a few geometric notions including “extreme points” that are useful
when we talk about the geometry of feasible regions and basic solutions.
Let x1 , x2 be points in Rd , and let S be a subset of Rd . A convex combination of x1 and
x is a point λx1 + (1 − λ)x2 for some 0 ≤ λ ≤ 1. The line segment between x1 and x2 is
2

the set of all convex combinations of them:

(4.39) [x1 , x2 ] = {x ∈ Rd : x = λx1 + (1 − λ)x2 , 0 ≤ λ ≤ 1}.

A subset S of Rd is called convex if the line segment between any two points in S is contained
in itself. See Figure 4.12. A point x is called an extreme point of S if x ∈ S and x cannot
be represented as a convex combination of two points of S different from x.
64 LP Algorithms

x2
x1 x1
x2

Figure 4.12: Convex and Nonconvex Sets

Proposition 4.21 The feasible region Ω is convex.

Proof. We use the representation Ω = {x ∈ RE : Ax ≤ b}. Let x1 , x2 be points of Ω, and

let x be a convex combination of x1 and x2 . Since
Ax = A(λx1 + (1 − λ)x2 ) = λAx1 + (1 − λ)Ax2
≤ λb + (1 − λ)b = b,

the set Ω is convex.

A solution x to a system of inequalities Ax ≤ b is said to be elementary if it is the unique

solution to A1 x = b1 , where A1 x ≤ b1 is the maximal subsystem of Ax ≤ b satisfied by x
with equality, i.e. determined uniquely by
( 1) ( 1)
A b
A= 2 ,b = , A1 x = b1 , A2 x < b2 .
A b2

Theorem 4.22 Let Ω = {x ∈ RE : Ax ≤ b}. Then a point x ∈ Ω is an extreme point if

and only if it is an elementary solution to Ax ≤ b.

Proof. Let x be an elementary solution. Suppose x be a convex combination of two

points x and x2 in Ω, i.e. x = λx1 + (1 − λ)x2 for some λ with 0 ≤ λ ≤ 1. Now pick up
1

any inequality Ai x ≤ bi from A1 x = b1 . We claim that both x1 and x2 must satisfy this
inequality with equality, since otherwise

Ai x = Ai (λx1 + (1 − λ)x2 ) = λAi x1 + (1 − λ)Ai x2

< λbi + (1 − λ)bi = bi ,

a contradiction to the choice of i. This means both x1 and x2 satisfy A1 x = b1 . Since x is

elementary, all x, x1 and x2 must be equal. This implies x is an extreme point.
The other direction is left for exercise.

Corollary 4.23 The notion of elementary solutions depends only on the set of solutions to
the given system.
4.9 Geometry of Pivots 65

Theorem 4.24 Every basic feasible solution to an LP in dictionary form is an extreme

point of its feasible region.

Proof. Consider an LP in dictionary form. Represent the constraints

xB = DxN , xE\{f,g} ≥ 0, xg = 1

as Ax ≤ b by using two opposite inequalities for each equality. Thus the feasible region is
{x : Ax ≤ b}.
Note that the basic solution x(B, g) is uniquely determined by xB = DxN , xg = 1 and
xN −g = 0. Clearly these equalities are contained in the maximal equality subsystem w.r.t.
x(B, g) of Ax ≤ b, and thus the subsystem must have a unique solution. This implies x(B, g)
is an elementary solution to Ax ≤ b. By Theorem 4.22, x(B, g) is an extreme point of the
feasible region.
66 LP Algorithms

4.10 Examples of Pivot Sequences

4.10.1 An example of cycling by the simplex method

maximize xf = 0 + x1 − 2x2 + x3
subject to x4 = 0 − 2x1 + x2 − x3
x5 = 0 − 3x1 − x2 − x3
x6 = 0 + 5x1 − 3x2 + 2x3
x1 , x2 , x3 , x4 , x5 , x6 ≥0

g 1 2 3 g 1 6 3
f 0 1 −2 1 f 0 −7/3 2/3 −1/3
4 0 −2 1 −1 ⇐= 4 0 −1/3 −1/3 −1/3
5 0 −3 −1 −1 5 0 −14/3 1/3 −5/3
6 0 5 −3 2 2 0 5/3 −1/3 2/3

⇓ ⇑

g 5 2 3 g 1 6 4
f 0 −1/3 −7/3 2/3 f 0 −2 1 1
4 0 2/3 5/3 −1/3 3 0 −1 −1 −3
1 0 −1/3 −1/3 −1/3 5 0 −3 2 5
6 0 −5/3 −14/3 1/3 2 0 1 −1 −2

⇓ ⇑

g 5 2 4 g 5 6 4
f 0 1 1 −2 f 0 2/3 −1/3 −7/3
3 0 2 5 −3 =⇒ 3 0 1/3 −5/3 −14/3
1 0 −1 −2 1 1 0 −1/3 2/3 5/3
6 0 −1 −3 −1 2 0 −1/3 −1/3 −1/3

Note: How can one find an example of cycling? By chance? Not quite. The key idea lies
in a geometric interpretation of the dual simplex method in the space of the primal problem.
4.10 Examples of Pivot Sequences 67

4.10.2 Simplex Method (Phase II) applied to Chateau ETH prob-

lem

maximize xf = 0 + 3x1 + 4x2 + 2x3

subject to x4 = 4 − 2x1
x5 = 8 − x1 − 2x3
x6 = 6 − 3x2 − x3
x1 , x2 , x3 , x4 , x5 , x6 ≥0

g 1 2 3
f 0 3 4 2
4 4 −2 0 0
5 8 −1 0 −2
6 6 0 −3 −1

g 1 2 5
f 8 2 4 −1
4 4 −2 0 0
3 4 −1/2 0 −1/2
6 2 1/2 −3 1/2

g 1 6 5
f 32/3 8/3 −4/3 −1/3
4 4 −2 0 0
3 4 −1/2 0 −1/2
2 2/3 1/6 −1/3 1/6

g 4 6 5
f 16 −4/3 −4/3 −1/3
1 2 −1/2 0 0
3 3 1/4 0 −1/2
2 1 −1/12 −1/3 1/6
68 LP Algorithms

4.10.3 Criss-Cross method applied to Chateau ETH problem

maximize xf = 0 + 3x1 + 4x2 + 2x3

subject to x4 = 4 − 2x1
x5 = 8 − x1 − 2x3
x6 = 6 − 3x2 − x3
x1 , x2 , x3 , x4 , x5 , x6 ≥0

g 1 2 3
f 0 3 4 2
4 4 −2 0 0
5 8 −1 0 −2
6 6 0 −3 −1

g 4 2 3
f 6 −3/2 4 2
1 2 −1/2 0 0
5 6 1/2 0 −2
6 6 0 −3 −1

g 4 6 3
f 14 −3/2 −4/3 2/3
1 2 −1/2 0 0
5 6 1/2 0 −2
2 2 0 −1/3 −1/3

g 4 6 2
f 18 −3/2 −2 −2
1 2 −1/2 0 0
5 −6 1/2 2 6
3 6 0 −1 −3

g 4 6 5
f 16 −4/3 −4/3 −1/3
1 2 −1/2 0 0
2 1 −1/12 −1/3 1/6
3 3 1/4 0 −1/2
4.10 Examples of Pivot Sequences 69

4.10.4 Criss-Cross method applied to a cycling example

maximize xf = 0 + x1 − 2x2 + x3
subject to x4 = 0 − 2x1 + x2 − x3
x5 = 0 − 3x1 − x2 − x3
x6 = 0 + 5x1 − 3x2 + 2x3
x1 , x2 , x3 , x4 , x5 , x6 ≥0

g 1 2 3
f 0 1 −2 1
4 0 −2 1 −1
5 0 −3 −1 −1
6 0 5 −3 2

g 4 2 3
f 0 −1/2 −3/2 1/2
1 0 −1/2 1/2 −1/2
5 0 3/2 −5/2 1/2
6 0 −5/2 −1/2 −1/2

g 4 2 1
f 0 −1 −1 −1
3 0 −1 1 −2
5 0 1 −2 −1
6 0 −2 −1 1
70 LP Algorithms

4.10.5 Simplex method with Bland’s rule applied to a cycling ex-

ample

maximize xf = 0 + x1 − 2x2 + x3
subject to x4 = 0 − 2x1 + x2 − x3
x5 = 0 − 3x1 − x2 − x3
x6 = 0 + 5x1 − 3x2 + 2x3
x1 , x2 , x3 , x4 , x5 , x6 ≥0

g 1 2 3
f 0 1 −2 1
4 0 −2 1 −1
5 0 −3 −1 −1
6 0 5 −3 2

g 4 2 3
f 0 −1/2 −3/2 1/2
1 0 −1/2 1/2 −1/2
5 0 3/2 −5/2 1/2
6 0 −5/2 −1/2 −1/2

g 4 2 1
f 0 −1 −1 −1
3 0 −1 1 −2
5 0 1 −2 −1
6 0 −2 −1 1
4.11 Visual Description of Pivot Algorithms 71

4.11 Visual Description of Pivot Algorithms

4.11.1 Simplex Method

g
f
⊕
..
.
⊕

↓ 9
g s g
f + f - · · · -
⊕ ⊕
.. ..
. .
⊕ ⊕
stop: optimal
↓ 9
g s g s
f + f +
⊕ ⊕ ⊕
.. .. ..
r . − r . .
⊕ ⊕ ⊕
ratio test stop: unbounded

↓ pivot on (r, s)

new feasible dictionary

72 LP Algorithms

4.11.2 Criss-Cross Method

g
f

↓ 9
g g
f f - · · · -
⊕
..
non-optimal .
⊕
stop: optimal

select : 9 min index r or s

g g s
f f +

r −

min s : ↓ ↓ 9 min r

g s g g s g s
f f f + f +
⊕
r − + r − - · · · - ..
. r −
⊕
stop:inconsistent stop:dual inconsistent

pivot ↓ ↓ pivot

new dictionary new dictionary