Algorithms R OBERT S EDGEWICK | K EVIN W AYNE

L INEAR P ROGRAMMING
‣ brewer’s problem
‣ simplex algorithm
‣ implementations
Algorithms
‣ reductions
F O U R T H E D I T I O N

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
Linear programming

What is it? Problem-solving model for optimal allocation of scarce

resources, among a number of competing activities that encompasses:
Shortest paths, maxflow, MST, matching, assignment, ... can take an entire
A x = b, 2-person zero-sum games, ... course on LP

maximize 13A + 23B

5A + 15B ≤ 480
subject
to the 4A + 4B ≤ 160
constraints
35A + 20B ≤ 1190

A , B ≥ 0

Why significant?
Fast commercial solvers available.
Ex: Delta claims that LP
Widely applicable problem-solving model. saves $100 million per year.

Key subroutine for integer programming solvers.

2
Applications

Agriculture. Diet problem.

Computer science. Compiler register allocation, data mining.
Electrical engineering. VLSI design, optimal clocking.
Energy. Blending petroleum products.
Economics. Equilibrium theory, two-person zero-sum games.
Environment. Water quality management.
Finance. Portfolio optimization.
Logistics. Supply-chain management.
Management. Hotel yield management.
Marketing. Direct mail advertising.
Manufacturing. Production line balancing, cutting stock.
Medicine. Radioactive seed placement in cancer treatment.
Operations research. Airline crew assignment, vehicle routing.
Physics. Ground states of 3-D Ising spin glasses.
Telecommunication. Network design, Internet routing.
Sports. Scheduling ACC basketball, handicapping horse races.
3
 L INEAR P ROGRAMMING
‣ brewer’s problem
‣ simplex algorithm
‣ implementations
Algorithms
‣ reductions

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu

Allocation of Resources by Linear Programming

by Robert Bland
Scientific American, Vol. 244, No. 6, June 1981
Toy LP example: brewer’s problem

Small brewery produces ale and beer.

Production limited by scarce resources: corn, hops, barley malt.

corn (480 lbs) hops (160 oz) malt (1190 lbs)

Recipes for ale and beer require different proportions of resources.

$13 profit per barrel $23 profit per barrel

5
 Toy LP example: brewer’s problem

Brewer’s problem: choose product mix to maximize profits.

34 barrels × 35 lbs malt = 1190 lbs
[ amount of available malt ]

ale beer corn hops malt profit

34 0 179 136 1190 $442

0 32 480 128 640 $736

19.5 20.5 405 160 1092.5 $725

goods are divisible

12 28 480 160 980 $800
(ok to produce
fractional barrels)
? ? > $800 ?

corn (480 lbs) hops (160 oz) malt (1190 lbs) $13 profit per barrel $23 profit per barrel
6
Brewer’s problem: linear programming formulation

Linear programming formulation.

Let A be the number of barrels of ale.
Let B be the number of barrels of beer.

ale beer

maximize 13A + 23B profits

5A + 15B ≤ 480 corn

subject
to the 4A + 4B ≤ 160 hops
constraints
35A + 20B ≤ 1190 malt

A , B ≥ 0

7
Brewer’s problem: feasible region

Inequalities define halfplanes; feasible region is a convex polygon.

hops malt
4A + 4B ≤ 160 35A + 20B ≤ 1190

(0, 32)

(12, 28)

corn
beer 5A + 15B ≤ 480
(26, 14)

(0, 0) ale (34, 0)

8
Brewer’s problem: objective function

hi
gh
er
pr
of
it?
(0, 32)

(12, 28)

13A + 23B = $1600

beer
(26, 14)

13A + 23B = $800

(0, 0) ale (34, 0)

13A + 23B = $442

9
7
Brewer’s problem: geometry

Optimal solution occurs at an extreme point.

intersection of 2 constraints in 2d

extreme point
(0, 32)

(12, 28)

beer
(26, 14)

(0, 0) ale (34, 0)

10
7
Standard form linear program

Goal. Maximize linear objective function of n nonnegative variables,

subject to m linear equations.
Input: real numbers aij, cj, bi. linear means no x2, xy, arccos(x), etc.

Output: real numbers xj.

primal problem (P) matrix version

maximize c1 x1 + c2 x2 + … + cn xn maximize cT x
a11 x1 + a12 x2 + … + a1n xn = b1
subject Ax = b
subject
a21 x1 + a22 x2 + … + a2n xn = b2 to the
to the
⋮ ⋮ ⋮ ⋮ ⋮ constraints x ≥ 0
constraints
am1 x1 + am2 x2 + … + amn xn = bm

x1 , x2 , … , xn ≥ 0

Caveat. No widely agreed notion of "standard form."

11
Converting the brewer’s problem to the standard form

Original formulation.
maximize 13A + 23B

5A + 15B ≤ 480
subject
to the 4A + 4B ≤ 160
constraints
35A + 20B ≤ 1190

A , B ≥ 0

Standard form.
Add variable Z and equation corresponding to objective function.
Add slack variable to convert each inequality to an equality.
Now a 6-dimensional problem.

maximize Z

13A + 23B − Z = 0
subject 5A + 15B + SC = 480
to the
constraints 4A + 4B + SH = 160

35A + 20B + SM = 1190

A , B , SC , SC , SM ≥ 0
12
Geometry

Inequalities define halfspaces; feasible region is a convex polyhedron.

A set is convex if for any two points a and b in the set, so is ½ (a + b).

An extreme point of a set is a point in the set that can't be written as

½ (a + b), where a and b are two distinct points in the set.
extreme
point

not convex convex

Warning. Don't always trust intuition in higher dimensions.

13
Geometry (continued)

Extreme point property. If there exists an optimal solution to (P),

then there exists one that is an extreme point.
Good news: number of extreme points to consider is finite.
Bad news : number of extreme points can be exponential!

local optima are global optima

(follows because objective function is linear
and feasible region is convex)

Greedy property. Extreme point optimal iff no better adjacent extreme point.
14
 L INEAR P ROGRAMMING
‣ brewer’s problem
‣ simplex algorithm
‣ implementations
Algorithms
‣ reductions

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
 L INEAR P ROGRAMMING
‣ brewer’s problem
‣ simplex algorithm
‣ implementations
Algorithms
‣ reductions

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
Simplex algorithm

Simplex algorithm. [George Dantzig, 1947]

Developed shortly after WWII in response to logistical problems,
including Berlin airlift.
Ranked as one of top 10 scientific algorithms of 20th century.

Generic algorithm. never decreasing objective function

Start at some extreme point.

Pivot from one extreme point to an adjacent one.
Repeat until optimal.

How to implement? Linear algebra.

17
 Simplex algorithm: basis

A basis is a subset of m of the n variables.

Basic feasible solution (BFS).

Set n – m nonbasic variables to 0, solve for remaining m variables.
Solve m equations in m unknowns.
If unique and feasible ⇒ BFS. basic feasible
BFS ⇔ extreme point. solution

{B, SH, SM } basic infeasible

{A, B, SM }
(0, 32) solution
(12, 28)

maximize Z {A, B, SH }

13A + 23B − Z = 0 (19.41, 25.53)

subject 5A + 15B + SC = 480 beer {A, B, SC }

to the (26, 14)
constraints 4A + 4B + SH = 160

35A + 20B + SM = 1190

A , B , SC , SH , SM ≥ 0
{SH, SM, SC } ale {A, SH, SC }
(0, 0) (34, 0)
18
 Simplex algorithm: initialization

maximize Z
basis = { SC, SH, SM }
13A + 23B − Z = 0 A=B=0
subject 5A + 15B + SC = 480 Z=0
to the SC = 480
constraints 4A + 4B + SH = 160
SH = 160
35A + 20B + SM = 1190 SM = 1190
A , B , SC , SH , SM ≥ 0

one basic variable per row

Initial basic feasible solution.

Start with slack variables { SC , SH , SM } as the basis.
no algebra needed
Set non-basic variables A and B to 0.
3 equations in 3 unknowns yields SC = 480, SH = 160, SM = 1190.

19
 Simplex algorithm: pivot 1

maximize Z pivot basis = { SC, SH, SM }

13A + 23B − Z = 0 A=B=0
subject 5A + 15B + SC = 480 Z=0
to the SC = 480
constraints 4A + 4B + SH = 160
SH = 160
35A + 20B + SM = 1190 SM = 1190
A , B , SC , SH , SM ≥ 0

substitute B = (1/15) (480 – 5A – SC) and add B into the basis which basic variable
does B replace?
(rewrite 2nd equation, eliminate B in 1st, 3rd, and 4th equations)

maximize Z basis = { B, SH, SM }

(16/3) A − (23/15) SC − Z = -736 A = SC = 0
subject Z = 736
(1/3) A + B + (1/15) SC = 32
to the B = 32
constraints (8/3) A − (4/15) SC + SH = 32
SH = 32
(85/3) A − (4/3) SC + SM = 550 SM = 550

A , B , SC , SH , SM ≥ 0

20
 Simplex algorithm: pivot 1
positive coefficient

maximize Z pivot basis = { SC, SH, SM }

13A + 23B − Z = 0 A=B=0
subject 5A + 15B + SC = 480 Z=0
to the SC = 480
constraints 4A + 4B + SH = 160
SH = 160
35A + 20B + SM = 1190 SM = 1190
A , B , SC , SH , SM ≥ 0

Q. Why pivot on column 2 (corresponding to variable B)?

Its objective function coefficient is positive.
(each unit increase in B from 0 increases objective value by $23)
Pivoting on column 1 (corresponding to A) also OK.

Q. Why pivot on row 2?

Preserves feasibility by ensuring RHS ≥ 0.
Minimum ratio rule: min { 480/15, 160/4, 1190/20 }.
21
 Simplex algorithm: pivot 2

maximize Z
pivot basis = { B, SH, SM }
(16/3) A − (23/15) SC − Z = -736 A = SC = 0
subject (1/3) A + B + (1/15) SC = 32 Z = 736
to the B = 32
constraints (8/3) A − (4/15) SC + SH = 32
SH = 32
(85/3) A − (4/3) SC + SM = 550 SM = 550
A , B , SC , SH , SM ≥ 0

substitute A = (3/8) (32 + (4/15) SC – SH ) and add A into the basis which basic variable
does A replace?
(rewrite 3rd equation, eliminate A in 1st, 2nd, and 4th equations)

maximize Z
basis = { A, B, SM }
− SC − 2 SH − Z = -800 SC = SH = 0
subject B + (1/10) SC + (1/8) SH = 28 Z = 800
to the B = 28
constraints A − (1/10) SC + (3/8) SH = 12
A = 12
− (25/6) SC − (85/8) SH + SM = 110 SM = 110

A , B , SC , SH , SM ≥ 0

22
 Simplex algorithm: optimality

Q. When to stop pivoting?

A. When no objective function coefficient is positive.

Q. Why is resulting solution optimal?

A. Any feasible solution satisfies current system of equations.
In particular: Z = 800 – SC – 2 SH
Thus, optimal objective value Z* ≤ 800 since SC , SH ≥ 0.
Current BFS has value 800 ⇒ optimal.

maximize Z
basis = { A, B, SM }
− SC − 2 SH − Z = -800 SC = SH = 0
subject B + (1/10) SC + (1/8) SH = 28 Z = 800
to the B = 28
constraints A − (1/10) SC + (3/8) SH = 12
A = 12
− (25/6) SC − (85/8) SH + SM = 110 SM = 110

A , B , SC , SH , SM ≥ 0

23
 L INEAR P ROGRAMMING
‣ brewer’s problem
‣ simplex algorithm
‣ implementations
Algorithms
‣ reductions

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
 L INEAR P ROGRAMMING
‣ brewer’s problem
‣ simplex algorithm
‣ implementations
Algorithms
‣ reductions

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
Simplex tableau

Encode standard form LP in a single Java 2D array.

maximize Z

13A + 23B − Z = 0
subject 5A + 15B + SC = 480
to the
constraints 4A + 4B + SH = 160

35A + 20B + SM = 1190

A , B , SC , SH , SM ≥ 0

5 15 1 0 0 480

4 4 0 1 0 160 m A I b

35 20 0 0 1 1190
1 c 0 0
13 23 0 0 0 0
n m 1
initial simplex tableaux

26
Simplex tableau

Simplex algorithm transforms initial 2D array into solution.

maximize Z

− SC − 2 SH − Z = -800
subject B + (1/10) SC + (1/8) SH = 28
to the
constraints A − (1/10) SC + (3/8) SH = 12

− (25/6) SC − (85/8) SH + SM = 110

A , B , SC , SH , SM ≥ 0

0 1 1/10 1/8 0 28

1 0 -1/10 3/8 0 12 m x*

0 0 -25/6 -85/8 1 110

1 ≤0 ≤0 -Z*
0 0 -1 -2 0 -800
n m 1
final simplex tableaux

27
Simplex algorithm: initial simplex tableaux

Construct the initial simplex tableau.

m A I b

1 c 0 0
n m 1

public class Simplex

{
constructor
private double[][] a; // simplex tableaux
private int m, n; // M constraints, N variables

public Simplex(double[][] A, double[] b, double[] c)

{
m = b.length;
n = c.length;
a = new double[m+1][m+n+1]; put A[][] into tableau
for (int i = 0; i < m; i++)
for (int j = 0; j < n; j++)
a[i][j] = A[i][j];
for (int j = n; j < m + n; j++) a[j-n][j] = 1.0; put I[][] into tableau

for (int j = 0; j < n; j++) a[m][j] = c[j]; put c[] into tableau
for (int i = 0; i < m; i++) a[i][m+n] = b[i]; put b[] into tableau
}

28
Simplex algorithm: Bland's rule
0 q m+n
Find entering column q using Bland's rule: 0

index of first column whose objective function

p +
coefficient is positive.

m +

private int bland()

{
for (int q = 0; q < m + n; q++) entering column q has positive
if (a[M][q] > 0) return q; objective function coefficient

optimal
return -1;
}

29
Simplex algorithm: min-ratio rule
0 q m+n
Find leaving row p using min ratio rule. 0

(Bland's rule: if a tie, choose first such row)

p +

m +

private int minRatioRule(int q)

{
int p = -1; leaving row
for (int i = 0; i < m; i++)
{ consider only
if (a[i][q] <= 0) continue; positive entries
else if (p == -1) p = i;
else if (a[i][m+n] / a[i][q] < a[p][m+n] / a[p][q])
row p has min
p = i; ratio so far
}
return p;
}

30
Simplex algorithm: pivot
0 q m+n
Pivot on element row p, column q. 0

p +

m +

public void pivot(int p, int q)

{
for (int i = 0; i <= m; i++)
for (int j = 0; j <= m+n; j++)
scale all entries but
if (i != p && j != q) row p and column q
a[i][j] -= a[p][j] * a[i][q] / a[p][q];

for (int i = 0; i <= m; i++)

if (i != p) a[i][q] = 0.0; zero out column q

for (int j = 0; j <= m+n; j++)

if (j != q) a[p][j] /= a[p][q]; scale row p
a[p][q] = 1.0;
}

31
Simplex algorithm: bare-bones implementation
0 q m+n
Execute the simplex algorithm. 0

p +

m +

public void solve()

{
while (true)
{
int q = bland(); entering column q (optimal if -1)
if (q == -1) break;

int p = minRatioRule(q); leaving row p (unbounded if -1)

if (p == -1) ...

pivot on row p, column q

pivot(p, q);
}
}

32
Simplex algorithm: running time

Remarkable property. In typical practical applications, simplex algorithm

terminates after at most 2 (m + n) pivots.

“ Yes. Most of the time it solved problems with m equations in 2m or 3m steps—

that was truly amazing. I certainly did not anticipate that it would turn out to
be so terrific. I had had no experience at the time with problems in higher
dimensions, and I didn't trust my geometrical intuition. For example, my
intuition told me that the procedure would require too many steps wandering
from one adjacent vertex to the next. In practice it takes few steps. In brief,
one's intuition in higher dimensional space is not worth a damn! Only now,
almost forty years from the time when the simplex method was first proposed,
are people beginning to get some insight into why it works as well as it does. ”
— George Dantzig 1984

33
Simplex algorithm: running time

Remarkable property. In typical practical applications, simplex algorithm

terminates after at most 2 (m + n) pivots.

Pivoting rules. Carefully balance the cost of finding an entering variable

with the number of pivots needed.
No pivot rule is known that is guaranteed to be polynomial.
Most pivot rules are known to be exponential (or worse) in worst-case.

Smoothed Analysis of Algorithms: Why the Simplex

Algorithm Usually Takes Polynomial Time

Daniel A. Spielman Shang-Hua Teng

Department of Mathematics Akamai Technologies Inc. and
M.I.T. Department of Computer Science
Cambridge, MA 02139 University of Illinois at Urbana-Champaign
[email protected] [email protected]

ABSTRACT

34
Simplex algorithm: degeneracy

Degeneracy. New basis, same extreme point.

"stalling" is common in practice

Cycling. Get stuck by cycling through different bases that all correspond
to same extreme point.
Doesn't occur in the wild.
Bland's rule guarantees finite # of pivots.
choose lowest valid index for
entering and leaving columns

35
Simplex algorithm: implementation issues

To improve the bare-bones implementation.

Avoid stalling. requires artful engineering

Maintain sparsity. requires fancy data structures

Numerical stability. requires advanced math

Detect infeasibility. run "phase I" simplex algorithm

Detect unboundedness. no leaving row

Best practice. Don't implement it yourself!

Basic implementations. Available in many programming environments.

Industrial-strength solvers. Routinely solve LPs with millions of variables.
Modeling languages. Simplify task of modeling problem as LP.

36
LP solvers: industrial strength

“ a benchmark production planning model solved using linear programming would have
taken 82 years to solve in 1988, using the computers and the linear programming
algorithms of the day. Fifteen years later—in 2003—this same model could be solved
in roughly 1 minute, an improvement by a factor of roughly 43 million. Of this, a factor
of roughly 1,000 was due to increased processor speed, whereas a factor of roughly
43,000 was due to improvements in algorithms! ”
— Designing a Digital Future
( Report to the President and Congress, 2010 )

37
Brief history

1939. Production, planning. [Kantorovich]

1947. Simplex algorithm. [Dantzig]
1947. Duality. [von Neumann, Dantzig, Gale-Kuhn-Tucker]
1947. Equilibrium theory. [Koopmans]
1948. Berlin airlift. [Dantzig]
1975. Nobel Prize in Economics. [Kantorovich and Koopmans]
1979. Ellipsoid algorithm. [Khachiyan]
1984. Projective-scaling algorithm. [Karmarkar]
1990. Interior-point methods. [Nesterov-Nemirovskii, Mehorta, ...]

Kantorovich George Dantzig von Neumann Koopmans Khachiyan Karmarkar

38
 L INEAR P ROGRAMMING
‣ brewer’s problem
‣ simplex algorithm
‣ implementations
Algorithms
‣ reductions

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
 L INEAR P ROGRAMMING
‣ brewer’s problem
‣ simplex algorithm
‣ implementations
Algorithms
‣ reductions

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
Reductions to standard form

Minimization problem. Replace min 13A + 15B with max – 13A – 15B.
≥ constraints. Replace 4A + 4B ≥ 160 with 4A + 4B – SH = 160, SH ≥ 0.
Unrestricted variables. Replace B with B = B0 – B1, B0 ≥ 0 , B1 ≥ 0.

nonstandard form

KBMBKBx2 13A + 15B

bm#D2+i iQ, 5A + 15B ≤ 480
4A + 4B ≥ 160
35A + 20B = 1190
A ≥ 0
B Bb mM`2bi`B+i2/
standard form

KtBKBx2 −13A − 15B0 + 15B1

bm#D2+i iQ, 5A + 15B0 − 15B1 + SC = 480
4A + 4B0 − 4B1 − SH = 160
35A + 20B0 − 20B1 = 1190
A B0 B1 SC SH ≥ 0
41
Modeling

Linear “programming” (1950s term) = reduction to LP (modern term).

Process of formulating an LP model for a problem.
Solution to LP for a specific problem gives solution to the problem.

1. Identify variables.
2. Define constraints (inequalities and equations).
3. Define objective function.
software usually performs
4. Convert to standard form. this step automatically

Examples.
Maxflow.
Shortest paths.
Bipartite matching.
Assignment problem.
2-person zero-sum games.
...
42
 maxflow problem maxflo
V
6 E
Maxflow problem (revisited)
8
0 1 2.0 LP formulation LP solution
0 2 3.0 Maximize x 35 + x 45
Input. Weighted digraph G,
1 3single
3.0 source s and single
subject tosink t.
the constraints
1 4 1.0
Goal. Find maximum flow 3 1.0s to t.
2 from 0 ! x 01 ! 2 x 01 = 2
2 4 1.0 0 ! x 02 ! 3 x 02 = 2
3 5 2.0 0 ! x 13 ! 3 x 13 = 1
4 5 3.0
0 ! x 14 ! 1 x 14 = 1
0 ! x 23 ! 1 x 23 = 1
capacities
0 ! x 24 ! 1 x 24 = 1
0 ! x 35 ! 2 x 35 = 2
maxflow problem 0 ! x 45maxflow
! 3 solution x 45 = 2
V x 01 = x 13 + x 14Max flow from 0 to
6 E 5
8 x 02 = x 23 + x 24 0->2 3.0 2.0
0 1 2.0 LP formulation LP solution
x 13 + x 23 = x 35 0->1 2.0 2.0
0 2 3.0 Maximize x 35 + x 45
1 3 3.0 x 14 + x 24 = x 45 1->4 1.0 1.0
subject to the constraints
1 4 1.0 1->3 3.0 1.0
2 3 1.0 0 ! x 01 ! 2 x 01 = 2 2->3 1.0 1.0
2 4 1.0 0 ! x 02 ! 3 x 02 = 2 2->4 1.0 1.0
3 5 2.0 0 ! x 13 ! 3 x 13 = 1 3->5 2.0 2.0
4 5 3.0
0 ! x 14 ! 1 x 14 = 1 4->5 3.0 2.0
0 ! x 23 ! 1 x 23 = 1 Max flow value: 4.0
capacities
0 ! x 24 ! 1 x 24 = 1
0 ! x 35 ! 2 Example ofxreducing
35 = 2 network flow to linear programming
0 ! x 45 ! 3 x 45 = 2
x 01 = x 13 + x 14 43
x 02 = x 23 + x 24
 maxflow problem maxflo
V
6 E
Modeling the maxflow 8problem as a linear program
0 1 2.0 LP formulation LP solution
0 2 3.0 Maximize x 35 + x 45
Variables. xvw = flow on edge
1 3 v→w.
3.0 subject to the constraints
1 4 1.0
Constraints. Capacity and 2 flow
3 1.0 conservation. 0 ! x 01 ! 2 x 01 = 2
2 problem
maxflow
Objective function. Net flow4 into
1.0 t. 0 ! x 02 ! 3 x 02 = 2 maxflow so
3 5 2.0 0 ! x 13 ! 3 x 13 = 1
V
4 65 3.0E Max
0 ! x 14 ! 1 x 14 = 1
8 0
0 1capacities
2.0 0 ! x 23 ! 1
LP formulation LP solution
x 23 = 1
0
0 2 3.0 0 ! x ! 1
Maximize x2435 + x 45 x 24 = 1
1 3 3.0 1
subject to the
0 ! x 35 ! 2constraints x 35 = 2
1 4 1.0 1
maxflow problem maxflow
00!!xx0145!!2 3 solution x 01x=452= 2
2 3 1.0 2
V 2 4 1.0 x001!=xx0213!+3x 14Max flow fromx 02 =02 to 5
6 2
E 3 5 2.0
8 x002!=xx1323!+3x 24 0->2 3.0 x2.0 13 = 1 3
0 1 2.0 LP4 formulation
5 3.0 LP solution
x013!+xx1423!=1x 35 0->1 2.0 x2.0 14 = 1 4
0 2 3.0 Maximize x 35 + x 45 capacity constraints
1 3 3.0 capacities x014!+xx2324!=1x 45 1->4 1.0 x1.0 23 = 1 Max
subject to the constraints
1 4 1.0 0 ! x 24 ! 1 1->3 3.0 x1.0
24 = 1
2 3 1.0 0 ! x 01 ! 2 x 01 =02! x ! 2 2->3 1.0 x1.0
35 35 = 2
2 4 1.0 0 ! x 02 ! 3 x 02 =02! x ! 3 2->4 1.0 x1.0
3 5 2.0 45 45 = 2
0 ! x 13 ! 3 x 13x= 1= x + x 3->5 2.0 2.0
4 5 3.0 01 13 14
0 ! x 14 ! 1 x 14 = 1 4->5 3.0 2.0
x 02 = x 23 + x 24 flow conservation
0 ! x 23 ! 1 x 23 = 1 Max flow value: 4.0
capacities x 13 + x 23 = x 35 constraints
0 ! x 24 ! 1 x 24 = 1
x 14 + x 24 =network
Example of reducing x 45 flow to linear programming
0 ! x 35 ! 2 x 35 = 2
0 ! x 45 ! 3 x 45 = 2
x 01 = x 13 + x 14 44
x 02 = x 23 + x 24
Maximum cardinality bipartite matching problem

Input. Bipartite graph.

Goal. Find a matching of maximum cardinality.

set of edges with no vertex appearing twice

Interpretation. Mutual preference constraints.

People to jobs.
Students to writing seminars.

Alice Adobe
Adobe, Apple, Google Alice, Bob, Dave
Bob Apple
Adobe, Apple, Yahoo Alice, Bob, Dave A B C D E F
Carol Google
Google, IBM, Sun Alice, Carol, Frank
Dave IBM
Adobe, Apple Carol, Eliza
Eliza Sun 0 1 2 3 4 5
IBM, Sun, Yahoo Carol, Eliza, Frank
Frank Yahoo
Google, Sun, Yahoo Bob, Eliza, Frank

matching of cardinality 6:
Example: job offers A–1, B–5, C–2, D–0, E–3, F–4
45
Maximum cardinality bipartite matching problem

LP formulation. One variable per pair.

Interpretation. xij = 1 if person i assigned to job j.

xA0 + xA1 + xA2 + xB0 + xB1 + xB5 + xC2 + xC3 + xC4

maximize
+ xD0 + xD1 + xE3 + xE4 + xE5 + xF2 + xF4 + xF5

at most one job per person at most one person per job
xA0 + xA1 + xA2 ≤1 xA0 + xB0 + xD0 ≤1

xB0 + xB1 + xB5 ≤1 xA1 + xB1 + xD1 ≤1

subject
xC2 + xC3 + xC4 ≤1 xA2 + xC2 + xF2 ≤1
to the
xD0 + xD1 ≤1 xC3 + xE3 ≤1
constraints
xE3 + xE4 + xE5 ≤1 xC4 + xE4 + xF4 ≤1

xF2 + xF4 + xF5 ≤1 xB5 + xE5 + xF5 ≤1

all xij ≥ 0

Theorem. [Birkhoff 1946, von Neumann 1953]

All extreme points of the above polyhedron have integer (0 or 1) coordinates.
Corollary. Can solve matching problem by solving LP. not usually so lucky!

46
Linear programming perspective

Q. Got an optimization problem?

Ex. Maxflow, bipartite matching, shortest paths, … [many, many, more]

Approach 1: Use a specialized algorithm to solve it.

Algorithms 4/e.
Vast literature on algorithms.

Approach 2: Use linear programming.

Many problems are easily modeled as LPs.
Commercial solvers can solve those LPs.
Might be slower than specialized solution
(but you might not care).

Got an LP solver? Learn to use it!

47
Universal problem-solving model (in theory)

Is there a universal problem-solving model?

Maxflow.
Shortest paths.
Bipartite matching.
Assignment problem. tractable
Multicommodity flow.
…
Two-person zero-sum games.
Linear programming.
…

Factoring intractable ?

NP-complete problems.
… see next lecture

Does P = NP? No universal problem-solving model exists unless P = NP.

48
 L INEAR P ROGRAMMING
‣ brewer’s problem
‣ simplex algorithm
‣ implementations
Algorithms
‣ reductions

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu
Algorithms R OBERT S EDGEWICK | K EVIN W AYNE

L INEAR P ROGRAMMING
‣ brewer’s problem
‣ simplex algorithm
‣ implementations
Algorithms
‣ reductions
F O U R T H E D I T I O N

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu