3.

The Revised Simplex Method

Javier Larrosa Albert Oliveras Enric Rodrguez-Carbonell Problem Solving and Constraint Programming (RPAR)

Session 3 p.1/34

Tableau Simplex Method

The simplex algorithm we have seen so far is called tableau simplex algorithm Some observations: At each iteration we update the tableau xB = B 1 b B 1 RxR for the new basis Note: as A contains a unit matrix of slack variables, this involves computing B 1 For pricing only one negative reduced cost is needed For ratio test, only the column of the chosen non-basic variable in the tableau and the current basic solution are needed

Session 3 p.2/34

Revised Simplex Method

Idea: do not keep representation of full tableau, only B 1 Advantages over the tableau version: Time and space are saved Errors due to oating-point are easier to control We will develop revised algorithm for LPs of the form
min z = cT x Ax = b x0

Session 3 p.3/34

Basic Solution
For any basis B , the values of basic variables can be expressed in terms of non-basic variables:
BxB + RxR = b BxB = b RxR xB = B 1 b B 1 RxR

By denition, basic solution corresponds to assigning null values to all non-basic variables Basic solution: := B 1 b

Session 3 p.4/34

Optimality Conditions (1)

Basic variables in terms of non-basic variables:
xB = B 1 b B 1 RxR

Cost function can be split: cT x = cT xB + cT xR , where R B

cT are the costs of basic variables, B cT are the costs of non-basic variables R

Expressing cost function in terms of non-basic variables

cT x = cT x B + cT x R = R B cT (B 1 b B 1 RxR ) + cT xR = B R cT B 1 b cT B 1 RxR + cT xR = R B B cT B 1 b + (cT cT B 1 R)xR B R B

Session 3 p.5/34

Optimality Conditions (2)

cT x = cT B 1 b + (cT cT B 1 R)xR B R B

Let aj be the column in A corresponding to variable xj . Let us dene reduced cost of xj as dj := cj cT B 1 aj B Note that dT = cT cT B 1 R, so cT x = cT B 1 b + dT xR B R R R B Thus this notion of reduced cost matches with that used in the tableau version of the simplex method

Session 3 p.6/34

Optimality Conditions (3)

cT x = cT B 1 b + dT xR , where dj = cj cT B 1 aj R B B

Value z of cost function at current basic solution: make xR = 0 in above equation, i.e., z := cT B 1 b B To avoid repeating computations: Let us dene the simplex multiplier as := (B T )1 cB , or equivalently, T := cT B 1 B Then dj = cj T aj and z = T b (note z = cT holds too) B Optimality condition: dj 0 for all j R

Session 3 p.7/34

Improving a Non-Optimal Solution (1)

Let us assume reduced cost of non-basic variable xq violates the optimality condition: dq < 0 Current value of xq is 0. We can improve by increasing only this value while sign constraints of basic variables are satised. Let t 0 be new value for xq . Then: (recall: the value of xp is null if p R and p = q )
xB (t) = B 1 b B 1 RxR = B 1 b B 1 aq t = tq

where = B 1 b and q = B 1 aq

Session 3 p.8/34

Improving a Non-Optimal Solution (2)

How much do we improve? How does the objective value change as a function of t?
z(t) = cT x(t) = cT xB (t) + cT xR (t) = R B cT xB (t) + cq t = B cT tcT q + cq t = B B cT tcT B 1 aq + cq t = B B z + tdq

Hence the improvement in cost is z = z(t) z = tdq

Session 3 p.9/34

Improving a Non-Optimal Solution (3)

How can we satisfy sign constraints of basic variables?
i Let i {1, ..., m}. Then i-th component of xB (t) is i tq i We need i tq 0 i If q 0 the constraint is satised for all t 0 i If q > 0 we need i i q

The best improvement is achieved with

i i := min{ i | q > 0} q

We say the p-th basic variable is blocking when = p Then q is the pivot

p . p q

Session 3 p.10/34

Improving a Non-Optimal Solution (4)

i 1. If = + (there is no i such that 1 i m and q > 0):

Value of objective function can be decreased innitely. LP is unbounded. 2. If < + and the p-th basic variable is blocking: When setting xq = sign of basic variables is respected
p In particular, the p-th component of xB (t) is p q = 0

We can make a basis change: xq enters the basis and xkp leaves, where B = (k1 , ..., km )

Session 3 p.11/34

Update
New basic indices: B = B {kp } {q}
i New basic solution: p = , i = i q if i = p

New basis: B = B + (aq akp )eT p

New basic costs: cT = cT + (cq ckp )eT B p B

p)

where eT = (0, ..., 0, 1 , 0, ..., 0)T p

m)

New objective value: z = z + dq

Session 3 p.12/34

Algorithmic Description
1. Initialization: Find an initial feasible basis B Compute B 1 , = B 1 b, z = cT B 2. Pricing: Compute = cT B 1 and dj = cj T aj . B If for all j R, dj 0 then return OPTIMAL Else let q be such that dq < 0. Compute q = B 1 aq
i 3. Ratio test: Compute I = {i | 1 i m, q > 0}. If I = then return UNBOUNDED Else compute = miniI ( ii ) and p such that = p p q q

4. Update:
B = B {kp } {q} i p = , i = i q if i = p B = B + (aq akp )eT p z = z + dq

Go to 2.

Session 3 p.13/34

Updating Matrix Inverse (1)

Actually what we really care about is B 1 , not B We need it for computing = cT B 1 and q = B 1 aq B Recomputing B 1 at each iteration is too expensive (e.g. cubic with Gaussian elimination!)
There is efcient way of computing B 1 using B 1

Session 3 p.14/34

Updating Matrix Inverse (2)

Let b1 , ..., bm be the columns of B Let a be such that a = Bv = m vi bi i=1 Let p be such that 1 p m
1 Ba = (b1 , . . . , bp1 , a, bp+1 , . . . , bm ). Want to compute Ba

Note vp = 0 as otherwise rank(Ba ) < m. So

a = v p bp +
i=p vi bi v1 vp

bp = ,
1 vp

1 vp

a+
vp+1 vp

i=p

vi vp vm vp

bi
T

Let T =

,...,

vp1 vp

,...,

E = (e1 , . . . , ep1 , , ep+1 , . . . , em )

1 Then Ba E = B = Ba = EB 1

Session 3 p.15/34

Updating Matrix Inverse (3)

Application to the simplex algorithm: a = aq , v = q , where xq is entering variable Thus updating inverse reuses already computed data! Assume initial basis is B0 (e.g., unit matrix I ). Then at the k -th iteration of the simplex algorithm 1 the inverse matrix is B 1 = Ek Ek1 E2 E1 B0 , where Ei is the E matrix of the i-th iteration Each E matrix can be stored compactly
1 Can represent B 1 as list Ek Ek1 E2 E1 plus B0 : Product Form of the Inverse (PFI)

When list is long, matrix is recomputed (reinversion) Other ways of representing B 1 : LU factoritzation

Session 3 p.16/34

Bounded Variables
We want to solve with revised simplex LPs of the form
min cT x Ax = b xu ( k uk +)

In practice internally variables translated so that = or = 0 to save arithmetic operations Variable xk is lower bounded if k > Variable xk is upper bounded if uk < + Variable xk is range if k > and uk < + Variable xk is xed if range and k = uk Variable xk is free if k = and uk = +

Session 3 p.17/34

Basic Solution
For any basis B , the values of basic variables can be expressed in terms of non-basic variables:
xB = B 1 b B 1 RxR

Values a non-basic variable xj can be assigned to: If lower bounded: k (it is at lower bound; xj L) If upper bounded: uk (it is at upper bound; xj U ) 0 (it is at zero level; xj Z ) If free: Basic solution:
:= B 1 b
jL

B 1 aj j
jU

B 1 aj uj

Session 3 p.18/34

Optimality Conditions
Cost function in terms of non-basic variables:
cT x = cT B 1 b + dR xR B

where dj = cj cT B 1 aj for all variable xj B If xj L: cannot improve if dj 0 If xj U : cannot improve if dj 0 If xj Z : cannot improve if dj = 0 Optimality condition: no improving non-basic variable

Session 3 p.19/34

Objective Function
Cost function in terms of non-basic variables:
cT x = cT B 1 b + dR xR B

where dj = cj cT B 1 aj for all variable xj B Value z of cost function at current basic solution:
z = cT B 1 b + B
jL

dj j +
jU

dj uj

Session 3 p.20/34

Improving a Non-Optimal Solution (1)

Let xq be non-basic variable that can improve objective value by increasing its value. This can happen when xq is lower bounded and xq L; or xq is free (so xq Z ) Since increasing xq can improve objective value: dq < 0 Let t 0 be difference of new value xq wrt old value
xB (t) = B 1 b B 1 RxR = B 1 b tB 1 aq = tq B 1 aj j jL B 1 aj uj jU

where = B 1 b and q = B 1 aq

Session 3 p.21/34

Improving a Non-Optimal Solution (2)

How does the objective value change as a function of t?
z(t) = cT x(t) = cT xB (t) + cT xR (t) = B R cT xB (t) + tcq + B
jL

c j j +
jU

cj uj =

cT B 1 b + B cT B 1 b + B

(cj cT B 1 aj )j + B
jL jU

(cj cT B 1 aj )uj + tcq tcT B 1 aq B B

dj j +
jL jU

dj uj + tcq tcT B 1 aq = B

z + tcq tcT B 1 aq = B z + tdq

Hence the improvement in cost is z = z(t) z = tdq

Session 3 p.22/34

Improving a Non-Optimal Solution (3)

i Let i {1, ..., m}. Then i-th component of xB (t) is i tq Let xki be the i-th basic variable. Let i := ki , i := uki i We need i i tq i i i If q > 0: i tq i i i If q < 0: i tq i

i i i q i i i q

t t

But we need xq (t) uq too! Best improvement achieved with

i i := min(uq q , min{ ii i | q > 0}, min{ ii i | q < 0}) q q

If = + we have unboundedness Else if = uq q we have a bound ip: no pivot needed!

Session 3 p.23/34

Improving a Non-Optimal Solution (4)

i i := min(uq q , min{ ii i | q > 0}, min{ ii i | q < 0}) q q

Assume = +, = uq q . Thus variable xq enters basis and variable xkp leaves

i If = min{ ii i | q > 0}, xkp leaves basis at lower bound q i If = min{ ii i | q < 0}, xkp leaves basis at upper bound q

Session 3 p.24/34

Improving a Non-Optimal Solution (5)

Let xq be non-basic variable that can improve objective value by decreasing its value. This can happen when
xq is upper bounded and xq U ; or xq is free (so xq Z )

Since decreasing xq can improve objective value: dq > 0 Let t 0 be difference of new value xq wrt old value Again xB (t) = tq , where = B 1 b and q = B 1 aq Again the improvement in cost is z = z(t) z = tdq

Session 3 p.25/34

Improving a Non-Optimal Solution (6)

i Let i {1, ..., m}. Then i-th component of xB (t) is i tq Let xki be the i-th basic variable. Let i := ki , i := uki i We need i i tq i i i If q > 0: i tq i i i If q < 0: i tq i

i i i q i i i q

t t

But we need q xq (t) too! Best improvement achieved with

i i := max(q uq , max{ ii i | q < 0}, max{ ii i | q > 0}) q q

If = we have unboundedness Else if = q uq we have a bound ip: no pivot needed!

Session 3 p.26/34

Improving a Non-Optimal Solution (7)

i i := max(q uq , max{ ii i | q < 0}, max{ ii i | q > 0}) q q

Assume = , = q uq . Thus variable xq enters basis and variable xkp leaves

i If = max{ ii i | q < 0}, xkp leaves basis at lower bound q i If = max{ ii i | q > 0}, xkp leaves basis at upper bound q

Session 3 p.27/34

Update
New objective value: z = z + dq If bound ip Flip status of xq (i.e., xq L xq U ) New basic solution: = q Else
New basic indices: B = B {kp } {q} i New basic solution: p = xq + , i = i q if i = p New basis inverse: B 1 = EB 1 If entering variable comes from lower bound L = L {xq } else U = U {xq } If leaving variable leaves to lower bound L = L {xkp } else U = U {xkp }

Session 3 p.28/34

Tableau vs. Revised Simplex

Time is saved: # Tableau: all dk , all k are computed  Revised: no. of non-basic variables xk for which dk , k are computed can be adjusted Space is saved: # Tableau: even if A is sparse tableau tends to get lled  Revised: sparsity of A can be exploited for storage and pivots can be chosen to represent B 1 compactly Better numerical behaviour: # Tableau: errors due to oating-point arithmetic accumulate at each pivoting step  Revised: reinversion (PFI representation of B 1 ) or refactorization (LU representation of B 1 ) for resetting

Session 3 p.29/34

Original vs. Bounds Simplex

Time is saved: # Original: no special treatment of bounds  Bounds: bound ips are much cheaper than pivoting steps in simplex iterations Space is saved: # Original: each bound constraint becomes a row  Bounds: bounds are stored cheaply in arrays

Session 3 p.30/34

Sensitivity Analysis
Data may be inaccurate (observations, estimations): small changes mean completely different solutions? Sensitivity analysis = study of the stability of optimal solution with respect to small changes in problem data Let B be a basis for
min cT x Ax = b x0 B gives an optimal feasible solution iff

Feasibility: B 1 b 0 Optimality: dT = cT cT B 1 R 0 B R R

Session 3 p.31/34

Changes in Cost Function

Assume ck changes to ck + k Feasibility is not affected If k R, only optimality for xk is affected:
0 ck + k cT B 1 ak k dk B

If k B and xk is p-th basic variable, for any j R

p 0 cj (cT + k eT )B 1 aj = dj k eT B 1 aj = dj k j p p B p k j dj

k
j

dj p j

p if j > 0,

k
j

dj p j

p if j < 0

d d p p Thus max{ j | j < 0} k min{ j | j > 0} p p

Session 3 p.32/34

Changes in Right-Hand Side (1)

Assume bp , the p-th component of b, changes to bp + p Optimality is not affected Let p be the p-th column of B 1 and = B 1 b. The basic solution is feasible iff
0 B 1 (b + p ep ) = + p p
i 0 i + p p for all 1 i m i i p p if p > 0, i i i p p if p < 0 i

i i i i Thus max{ p | p > 0} p min{ p | p < 0} i i

Session 3 p.33/34

Changes in Right-Hand Side (2)

Assume bi , the i-th component of b, changes to bi + i , so that the optimal basis does not change How does this affect the objective function? Value of objective function at basic solution is
z(b) = cT B 1 b = T b,

where T = cT B 1 are simplex multipliers B

z(b + i ei ) z(b) = i T ei = i i

Can be used to determine price/benet of changing a resource, capacity, limit, etc. If xk is slack of i-th row, then dk is called shadow price. Note that dk = 0 cT B 1 ei = T ei = i . B

Session 3 p.34/34