Algorithms R OBERT S EDGEWICK | K EVIN W AYNE

Linear programming

What is it? Problem-solving model for optimal allocation of scarce

resources, among a number of competing activities that encompasses:
・Shortest paths, maxflow, MST, matching, assignment, ... can take an entire

・A x = b, 2-person zero-sum games, ... course on LP

L INEAR P ROGRAMMING
maximize 13A + 23B
‣ brewer’s problem 5A + 15B ≤ 480
subject
‣ simplex algorithm to the 4A + 4B ≤ 160

‣ implementations constraints
Algorithms 35A + 20B ≤ 1190

‣ reductions
F O U R T H E D I T I O N
A , B ≥ 0

R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu Why significant?

・Fast commercial solvers available. Ex: Delta claims that LP
・Widely applicable problem-solving model. saves $100 million per year.

・Key subroutine for integer programming solvers.

2

Applications

Agriculture. Diet problem.

Computer science. Compiler register allocation, data mining.
Electrical engineering. VLSI design, optimal clocking.
Energy. Blending petroleum products.
Economics. Equilibrium theory, two-person zero-sum games. L INEAR P ROGRAMMING
Environment. Water quality management.
Finance. Portfolio optimization. ‣ brewer’s problem
Logistics. Supply-chain management.
‣ simplex algorithm
Management. Hotel yield management.
‣ implementations
Marketing. Direct mail advertising. Algorithms
Manufacturing. Production line balancing, cutting stock. ‣ reductions
Medicine. Radioactive seed placement in cancer treatment.
R OBERT S EDGEWICK | K EVIN W AYNE
Operations research. Airline crew assignment, vehicle routing. http://algs4.cs.princeton.edu

Physics. Ground states of 3-D Ising spin glasses.

Telecommunication. Network design, Internet routing. Allocation of Resources by Linear Programming
by Robert Bland
Sports. Scheduling ACC basketball, handicapping horse races. Scientific American, Vol. 244, No. 6, June 1981
3
Toy LP example: brewer’s problem Toy LP example: brewer’s problem

Small brewery produces ale and beer. Brewer’s problem: choose product mix to maximize profits.
・Production limited by scarce resources: corn, hops, barley malt. 34 barrels × 35 lbs malt = 1190 lbs
[ amount of available malt ]

ale beer corn hops malt profit

34 0 179 136 1190 $442

0 32 480 128 640 $736

corn (480 lbs) hops (160 oz) malt (1190 lbs)
19.5 20.5 405 160 1092.5 $725
goods are
divisible 12 28 480 160 980 $800

・ Recipes for ale and beer require different proportions of resources.

? ? > $800 ?

$13 profit per barrel $23 profit per barrel

corn (480 lbs) hops (160 oz) malt (1190 lbs) $13 profit per barrel $23 profit per barrel
5 6

Brewer’s problem: linear programming formulation Brewer’s problem: feasible region

Linear programming formulation. Inequalities define halfplanes; feasible region is a convex polygon.
・Let A be the number of barrels of ale.
・Let B be the number of barrels of beer. hops
4A + 4B ≤ 160
malt
35A + 20B ≤ 1190

ale beer

maximize 13A + 23B profits

5A + 15B ≤ 480 corn (0, 32)

subject
to the 4A + 4B ≤ 160 hops
(12, 28)
constraints
35A + 20B ≤ 1190 malt

A , B ≥ 0
corn
beer 5A + 15B ≤ 480
(26, 14)

(0, 0) ale (34, 0)

7 8
Brewer’s problem: objective function Brewer’s problem: geometry

Optimal solution occurs at an extreme point.

intersection of 2 constraints in 2d

hi
gh
er
pr
of
it? extreme point
(0, 32) (0, 32)

(12, 28) (12, 28)

13A + 23B = $1600

beer beer
(26, 14) (26, 14)

13A + 23B = $800

(0, 0) ale (34, 0) (0, 0) ale (34, 0)

13A + 23B = $442

9
7 10
7

Standard form linear program Converting the brewer’s problem to the standard form

Goal. Maximize linear objective function of n nonnegative variables, Original formulation.

maximize 13A + 23B
subject to m linear equations.
5A + 15B ≤ 480
・Input: real numbers a , c , b . ij j i linear means no x2, xy, arccos(x), etc. subject
to the 4A + 4B ≤ 160
・Output: real numbers x . j constraints
35A + 20B ≤ 1190

A , B ≥ 0

primal problem (P) matrix version

Standard form.
maximize c1 x1 + c2 x2 + … + cn xn maximize cT x
・Add variable Z and equation corresponding to objective function.
subject
a11 x1 + a12 x2 + … + a1n xn = b1
subject Ax = b ・Add slack variable to convert each inequality to an equality.
to the
a21 x1 + a22 x2 + … + a2n xn = b2 to the
constraints x ≥ 0
・Now a 6-dimensional problem.
constraints ⋮ ⋮ ⋮ ⋮ ⋮
am1 x1 + am2 x2 + … + amn xn = bm maximize Z

13A + 23B − Z = 0
x1 , x2 , … , xn ≥ 0
subject 5A + 15B + SC = 480
to the
constraints 4A + 4B + SH = 160

35A + 20B + SM = 1190

Caveat. No widely agreed notion of "standard form."
A , B , SC , SC , SM ≥ 0
11 12
Geometry Geometry (continued)

Inequalities define halfspaces; feasible region is a convex polyhedron. Extreme point property. If there exists an optimal solution to (P),
then there exists one that is an extreme point.

A set is convex if for any two points a and b in the set, so is ½ (a + b). ・Good news: number of extreme points to consider is finite.
・Bad news : number of extreme points can be exponential!
An extreme point of a set is a point in the set that can't be written as
½ (a + b), where a and b are two distinct points in the set.
extreme
point
local optima are global optima
(follows because objective function is linear
and feasible region is convex)

not convex convex

Warning. Don't always trust intuition in higher dimensions. Greedy property. Extreme point optimal iff no better adjacent extreme point.
13 14

L INEAR P ROGRAMMING L INEAR P ROGRAMMING

‣ brewer’s problem ‣ brewer’s problem
‣ simplex algorithm ‣ simplex algorithm
‣ implementations ‣ implementations
Algorithms Algorithms
‣ reductions ‣ reductions

R OBERT S EDGEWICK | K EVIN W AYNE R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu http://algs4.cs.princeton.edu
 Simplex algorithm Simplex algorithm: basis

Simplex algorithm. [George Dantzig, 1947] A basis is a subset of m of the n variables.

・Developed shortly after WWII in response to logistical problems,
including Berlin airlift. Basic feasible solution (BFS).
・Ranked as one of top 10 scientific algorithms of 20th century. ・Set n – m nonbasic variables to 0, solve for remaining m variables.
・Solve m equations in m unknowns.
Generic algorithm. never decreasing objective function
・If unique and feasible ⇒ BFS. basic feasible

・ Start at some extreme point. ・BFS ⇔ extreme point. solution

・Pivot from one extreme point to an adjacent one. {B, SH, SM } basic infeasible

・Repeat until optimal.

{A, B, SM }
(0, 32) solution
(12, 28)

maximize Z {A, B, SH }
How to implement? Linear algebra. 13A + 23B − Z = 0 (19.41, 25.53)

subject 5A + 15B + SC = 480 beer {A, B, SC }

to the (26, 14)
constraints 4A + 4B + SH = 160

35A + 20B + SM = 1190

A , B , SC , SH , SM ≥ 0
{SH, SM, SC } ale {A, SH, SC }
(0, 0) (34, 0)
17 18

Simplex algorithm: initialization Simplex algorithm: pivot 1

maximize Z maximize Z pivot

basis = { SC, SH, SM } basis = { SC, SH, SM }
13A + 23B − Z = 0 A=B=0 13A + 23B − Z = 0 A=B=0
subject 5A + 15B + SC = 480 Z=0 subject 5A + 15B + SC = 480 Z=0
to the SC = 480 to the SC = 480
constraints 4A + 4B + SH = 160 constraints 4A + 4B + SH = 160
SH = 160 SH = 160
35A + 20B + SM = 1190 SM = 1190 35A + 20B + SM = 1190 SM = 1190
A , B , SC , SH , SM ≥ 0 A , B , SC , SH , SM ≥ 0

substitute B = (1/15) (480 – 5A – SC) and add B into the basis which basic variable
does B replace?
(rewrite 2nd equation, eliminate B in 1st, 3rd, and 4th equations)
one basic variable per row

Initial basic feasible solution.

・Start with slack variables { S , S , S } as the basis.
C H M
no algebra needed
maximize Z basis = { B, SH, SM }

・Set non-basic variables A and B to 0. (16/3) A − (23/15) SC − Z = -736 A = SC = 0

・3 equations in 3 unknowns yields S = 480, S = 160, S C H M = 1190.

subject
to the
(1/3) A + B + (1/15) SC = 32 Z = 736
B = 32
constraints (8/3) A − (4/15) SC + SH = 32
SH = 32
(85/3) A − (4/3) SC + SM = 550 SM = 550

A , B , SC , SH , SM ≥ 0

19 20
 Simplex algorithm: pivot 1 Simplex algorithm: pivot 2
positive coefficient

maximize Z maximize Z
pivot basis = { SC, SH, SM } pivot basis = { B, SH, SM }
13A + 23B − Z = 0 A=B=0 (16/3) A − (23/15) SC − Z = -736 A = SC = 0
subject Z=0 subject (1/3) A + B + (1/15) SC = 32 Z = 736
5A + 15B + SC = 480
to the SC = 480 to the B = 32
4A + 4B + SH = 160 constraints (8/3) A − (4/15) SC + SH = 32
constraints SH = 160 SH = 32
35A + 20B + SM = 1190 SM = 1190 (85/3) A − (4/3) SC + SM = 550 SM = 550
A , B , SC , SH , SM ≥ 0 A , B , SC , SH , SM ≥ 0

substitute A = (3/8) (32 + (4/15) SC – SH ) and add A into the basis which basic variable
does A replace?
(rewrite 3rd equation, eliminate A in 1st, 2nd, and 4th equations)
Q. Why pivot on column 2 (corresponding to variable B)?
・Its objective function coefficient is positive.
(each unit increase in B from 0 increases objective value by $23) maximize Z
basis = { A, B, SM }

・ Pivoting on column 1 (corresponding to A) also OK.

subject
− SC − 2 SH − Z = -800 SC = SH = 0
B + (1/10) SC + (1/8) SH = 28 Z = 800
to the B = 28
Q. Why pivot on row 2? constraints A − (1/10) SC + (3/8) SH = 12
A = 12

・ Preserves feasibility by ensuring RHS ≥ 0. − (25/6) SC − (85/8) SH + SM = 110 SM = 110

・Minimum ratio rule: min { 480/15, 160/4, 1190/20 }. A , B , SC , SH , SM ≥ 0

21 22

Simplex algorithm: optimality

Q. When to stop pivoting?

A. When no objective function coefficient is positive.

Q. Why is resulting solution optimal?

A. Any feasible solution satisfies current system of equations. L INEAR P ROGRAMMING
・ In particular: Z = 800 – SC – 2 SH

・Thus, optimal objective value Z* ≤ 800 since S C, SH ≥ 0. ‣ brewer’s problem

・Current BFS has value 800 ⇒ optimal. ‣ simplex algorithm
‣ implementations
Algorithms
maximize Z ‣ reductions
basis = { A, B, SM }
− SC − 2 SH − Z = -800 SC = SH = 0
R OBERT S EDGEWICK | K EVIN W AYNE
subject B + (1/10) SC + (1/8) SH = 28 Z = 800
to the http://algs4.cs.princeton.edu
B = 28
constraints A − (1/10) SC + (3/8) SH = 12
A = 12
− (25/6) SC − (85/8) SH + SM = 110 SM = 110

A , B , SC , SH , SM ≥ 0

23
 Simplex tableau

Encode standard form LP in a single Java 2D array.

maximize Z

13A + 23B − Z = 0
subject
L INEAR P ROGRAMMING to the
5A + 15B + SC = 480

constraints 4A + 4B + SH = 160

‣ brewer’s problem 35A + 20B + SM = 1190

A , B , SC , SH , SM ≥ 0
‣ simplex algorithm
‣ implementations
Algorithms 5 15 1 0 0 480
‣ reductions
4 4 0 1 0 160 m A I b
R OBERT S EDGEWICK | K EVIN W AYNE
35 20 0 0 1 1190
http://algs4.cs.princeton.edu 1 c 0 0
13 23 0 0 0 0
n m 1
initial simplex tableaux

26

Simplex tableau Simplex algorithm: initial simplex tableaux

Simplex algorithm transforms initial 2D array into solution. Construct the initial simplex tableau.
m A I b
maximize Z

− SC − 2 SH − Z = -800 1 c 0 0
subject B + (1/10) SC + (1/8) SH = 28 n m 1
to the
constraints A − (1/10) SC + (3/8) SH = 12
public class Simplex
− (25/6) SC − (85/8) SH + SM = 110 {
constructor
private double[][] a; // simplex tableaux
A , B , SC , SH , SM ≥ 0 private int m, n; // M constraints, N variables

public Simplex(double[][] A, double[] b, double[] c)

{
0 1 1/10 1/8 0 28 m = b.length;
n = c.length;
1 0 -1/10 3/8 0 12 m x* a = new double[m+1][m+n+1]; put A[][] into tableau
for (int i = 0; i < m; i++)
0 0 -25/6 -85/8 1 110 for (int j = 0; j < n; j++)
1 ≤0 ≤0 -Z* a[i][j] = A[i][j];
0 0 -1 -2 0 -800 put I[][] into tableau
for (int j = n; j < m + n; j++) a[j-n][j] = 1.0;
n m 1 put c[] into tableau
for (int j = 0; j < n; j++) a[m][j] = c[j];
final simplex tableaux for (int i = 0; i < m; i++) a[i][m+n] = b[i]; put b[] into tableau
}

27 28
Simplex algorithm: Bland's rule Simplex algorithm: min-ratio rule
0 q m+n 0 q m+n
Find entering column q using Bland's rule: 0 Find leaving row p using min ratio rule. 0

index of first column whose objective function (Bland's rule: if a tie, choose first such row)
p + p +
coefficient is positive.

m + m +

private int minRatioRule(int q)

private int bland() {
{ leaving row
int p = -1;
for (int q = 0; q < m + n; q++) entering column q has positive for (int i = 0; i < m; i++)
if (a[M][q] > 0) return q; objective function coefficient { consider only
optimal if (a[i][q] <= 0) continue; positive entries
return -1; else if (p == -1) p = i;
}
else if (a[i][m+n] / a[i][q] < a[p][m+n] / a[p][q])
row p has min
p = i; ratio so far
}
return p;
}

29 30

Simplex algorithm: pivot Simplex algorithm: bare-bones implementation

0 q m+n 0 q m+n
Pivot on element row p, column q. 0 Execute the simplex algorithm. 0

p + p +

m + m +

public void pivot(int p, int q)

public void solve()
{
{
for (int i = 0; i <= m; i++)
while (true)
for (int j = 0; j <= m+n; j++)
scale all entries but {
if (i != p && j != q) row p and column q int q = bland(); entering column q (optimal if -1)
a[i][j] -= a[p][j] * a[i][q] / a[p][q];
if (q == -1) break;

for (int i = 0; i <= m; i++)

int p = minRatioRule(q); leaving row p (unbounded if -1)
if (i != p) a[i][q] = 0.0; zero out column q
if (p == -1) ...

for (int j = 0; j <= m+n; j++) pivot on row p, column q

scale row p pivot(p, q);
if (j != q) a[p][j] /= a[p][q];
}
a[p][q] = 1.0;
}
}

31 32
Simplex algorithm: running time Simplex algorithm: running time

Remarkable property. In typical practical applications, simplex algorithm Remarkable property. In typical practical applications, simplex algorithm
terminates after at most 2 (m + n) pivots. terminates after at most 2 (m + n) pivots.

Pivoting rules. Carefully balance the cost of finding an entering variable

“ Yes. Most of the time it solved problems with m equations in 2m or 3m steps— with the number of pivots needed.
that was truly amazing. I certainly did not anticipate that it would turn out to ・No pivot rule is known that is guaranteed to be polynomial.
be so terrific. I had had no experience at the time with problems in higher ・Most pivot rules are known to be exponential (or worse) in worst-case.
dimensions, and I didn't trust my geometrical intuition. For example, my
intuition told me that the procedure would require too many steps wandering
from one adjacent vertex to the next. In practice it takes few steps. In brief,
one's intuition in higher dimensional space is not worth a damn! Only now, Smoothed Analysis of Algorithms: Why the Simplex
Algorithm Usually Takes Polynomial Time
almost forty years from the time when the simplex method was first proposed,
are people beginning to get some insight into why it works as well as it does. ” Daniel A. Spielman Shang-Hua Teng
Department of Mathematics Akamai Technologies Inc. and
M.I.T. Department of Computer Science
— George Dantzig 1984 Cambridge, MA 02139 University of Illinois at Urbana-Champaign
[email protected] [email protected]

ABSTRACT

33 34

1. INTRODUCTION

Simplex algorithm: degeneracy Simplex algorithm: implementation issues

Degeneracy. New basis, same extreme point. To improve the bare-bones implementation.

"stalling" is common in practice

・Avoid stalling. requires artful engineering

・Maintain sparsity. requires fancy data structures

・Numerical stability. requires

1.1advanced math
Background

・Detect infeasibility. run "phase I" simplex algorithm

・Detect unboundedness. no leaving row

Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
Best practice. Don't implement it yourself!
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
STOC’01, July 6-8, 2001, Hersonissos, Crete, Greece.
Copyright 2001 ACM 1-58113-349-9/01/0007 ... 5.00.

Basic implementations. Available in many

296 programming environments.
Cycling. Get stuck by cycling through different bases that all correspond Industrial-strength solvers. Routinely solve LPs with millions of variables.
to same extreme point. Modeling languages. Simplify task of modeling problem as LP.
・ Doesn't occur in the wild.
・Bland's rule guarantees finite # of pivots.
choose lowest valid index for
entering and leaving columns

35 36
LP solvers: industrial strength Brief history

1939. Production, planning. [Kantorovich]

1947. Simplex algorithm. [Dantzig]
1947. Duality. [von Neumann, Dantzig, Gale-Kuhn-Tucker]
“ a benchmark production planning model solved using linear programming would have 1947. Equilibrium theory. [Koopmans]
taken 82 years to solve in 1988, using the computers and the linear programming 1948. Berlin airlift. [Dantzig]
algorithms of the day. Fifteen years later—in 2003—this same model could be solved 1975. Nobel Prize in Economics. [Kantorovich and Koopmans]
in roughly 1 minute, an improvement by a factor of roughly 43 million. Of this, a factor 1979. Ellipsoid algorithm. [Khachiyan]
of roughly 1,000 was due to increased processor speed, whereas a factor of roughly 1984. Projective-scaling algorithm. [Karmarkar]
43,000 was due to improvements in algorithms! ” 1990. Interior-point methods. [Nesterov-Nemirovskii, Mehorta, ...]
— Designing a Digital Future
( Report to the President and Congress, 2010 )

Kantorovich George Dantzig von Neumann Koopmans Khachiyan Karmarkar

37 38

L INEAR P ROGRAMMING L INEAR P ROGRAMMING

‣ brewer’s problem ‣ brewer’s problem
‣ simplex algorithm ‣ simplex algorithm
‣ implementations ‣ implementations
Algorithms Algorithms
‣ reductions ‣ reductions

R OBERT S EDGEWICK | K EVIN W AYNE R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu http://algs4.cs.princeton.edu
Reductions to standard form Modeling

Minimization problem. Replace min 13A + 15B with max – 13A – 15B. Linear “programming” (1950s term) = reduction to LP (modern term).
≥ constraints. Replace 4A + 4B ≥ 160 with 4A + 4B – SH = 160, SH ≥ 0. ・Process of formulating an LP model for a problem.
Unrestricted variables. Replace B with B = B0 – B1, B0 ≥ 0 , B1 ≥ 0. ・Solution to LP for a specific problem gives solution to the problem.
nonstandard form
1. Identify variables.
KBMBKBx2 13A + 15B 2. Define constraints (inequalities and equations).
bm#D2+i iQ, 5A + 15B ≤ 480 3. Define objective function.
4A + 4B ≥ 160 software usually performs
4. Convert to standard form. this step automatically
35A + 20B = 1190
A ≥ 0
Examples.
B Bb mM`2bi`B+i2/
standard form
・Maxflow.
KtBKBx2 −13A − 15B0 + 15B1
・Shortest paths.
bm#D2+i iQ, 5A + 15B0 − 15B1 + SC = 480
・Bipartite matching.
4A + 4B0 − 4B1 − SH = 160
・Assignment problem.
35A + 20B0 − 20B1 = 1190
・2-person zero-sum games.
...
A B0 B1 SC SH ≥ 0
41 42

maxflow problem maxflow solution maxflow problem maxflo

V V
E 6 Max flow from 0 to 5 E 6
Maxflow problem (revisited)
8 Modeling
0->2 3.0 2.0 the maxflow 8problem as a linear program
0 1 2.0 LP formulation LP solution 0 1 2.0 LP formulation LP solution
0->1 2.0 2.0
0 2 3.0 Maximize x 35 + x 45 0 2 3.0 Maximize x 35 + x 45
Input. Weighted digraph G,
1 3single
3.0 source s and single t. 1->4 1.0 1.0 xvw = flow on edge
1 3 v→w.
subject tosink
the constraints Variables. 3.0 subject to the constraints
1 4 1.0 1->3 3.0 1.0 1 4 1.0
Goal. Find maximum flow 3 1.0s to t.
2 from 0 ! x 01 ! 2 x 01 = 2 2->3
Constraints.
1.0 1.0
Capacity and flow
2 3 1.0 conservation. 0 ! x 01 ! 2 x 01 = 2
2 4 1.0 0 ! x 02 ! 3 x 02 = 2 2->4 Objective
1.0 1.0 2 problem
maxflow
function. Net flow 4 into
1.0 t. 0 ! x 02 ! 3 x 02 = 2 maxflow so
3 5 2.0 0 ! x 13 ! 3 x 13 = 1 3 5 2.0 0 ! x 13 ! 3 x 13 = 1
3->5 2.0 2.0 V
4 5 3.0 4 65 3.0E Max
0 ! x 14 ! 1 x 14 = 1 4->5 3.0 2.0 0 ! x 14 ! 1 x 14 = 1
8 0
0 ! x 23 ! 1 x 23 = 1 Max flow value: 4.0 0 1capacities
2.0 0 ! x 23 ! 1
LP formulation LP solution
x 23 = 1
capacities 0
0 ! x 24 ! 1 x 24 = 1 0 2 3.0 0 ! xx2435!
Maximize + 1x 45 x 24 = 1
1 3 3.0 1
0 ! x 35 ! 2 x 35 = 2 subject
0! to xthe constraints
35 ! 2 x 35 = 2
1 4 1.0 1
maxflow problem 0 ! x 45maxflow
! 3 solution x 45 = 2 maxflow problem 00!!xx0145! maxflow
!2 3 solution x 01x=452= 2
2 3 1.0 2
V x 01 = x 13 + x 14Max flow from 0 to V 2 4 1.0 x001!=xx0213!+3x 14Max flow from
x 02 =02 to 5
6 5 6 2
E E 3 5 2.0
8 x 02 = x 23 + x 24 0->2 3.0 2.0 8 x002!=xx1323!+3x 24 0->2 3.0 x2.013 = 1 3
0 1 2.0 LP formulation LP solution 0 1 2.0 LP4 formulation
5 3.0 LP solution
x 13 + x 23 = x 35 0->1 2.0 2.0 x013!+xx1423!=1x 35 0->1 2.0 x2.014 = 1 4
0 2 3.0 Maximize x 35 + x 45 0 2 3.0 Maximize x 35 + x 45 capacity constraints
1 3 3.0 x 14 + x 24 = x 45 1->4 1.0 1.0 1 3 3.0 capacities x014!+xx2324!=1x 45 1->4 1.0 x1.023 = 1 Max
subject to the constraints subject to the constraints
1 4 1.0 1->3 3.0 1.0 1 4 1.0 0 ! x 24 ! 1 1->3 3.0 x1.0
24 = 1
2 3 1.0 0 ! x 01 ! 2 x 01 = 2 2->3 1.0 1.0 2 3 1.0 0 ! x 01 ! 2 x 01 =02! x ! 2 2->3 1.0 x1.0
35 35 = 2
2 4 1.0 0 ! x 02 ! 3 x 02 = 2 2->4 1.0 1.0 2 4 1.0 0 ! x 02 ! 3 x 02 =02! x ! 3 2->4 1.0 x1.0
3 5 2.0 3 5 2.0 45 45 = 2
0 ! x 13 ! 3 x 13 = 1 3->5 2.0 2.0 0 ! x 13 ! 3 x 13x= 1= x + x 3->5 2.0 2.0
4 5 3.0 4 5 3.0 01 13 14
0 ! x 14 ! 1 x 14 = 1 4->5 3.0 2.0 0 ! x 14 ! 1 x 14 = 1 4->5 3.0 2.0
x 02 = x 23 + x 24 flow conservation
0 ! x 23 ! 1 x 23 = 1 Max flow value: 4.0 0 ! x 23 ! 1 x 23 = 1
capacities capacities x 13 + x 23 = x 35 Max flow value: 4.0
constraints
0 ! x 24 ! 1 x 24 = 1 0 ! x 24 ! 1 x 24 = 1
x + x = x
0 ! x 35 ! 2 Example ofxreducing
35 = 2 network flow to linear programming 0 ! x 35 ! 2 Example ofxreducing
14
35 = 2
24 network
45 flow to linear programming
0 ! x 45 ! 3 x 45 = 2 0 ! x 45 ! 3 x 45 = 2
x 01 = x 13 + x 14 43
x 01 = x 13 + x 14 44
x 02 = x 23 + x 24 x 02 = x 23 + x 24
Maximum cardinality bipartite matching problem Maximum cardinality bipartite matching problem

Input. Bipartite graph. LP formulation. One variable per pair.

Goal. Find a matching of maximum cardinality. Interpretation. xij = 1 if person i assigned to job j.

set of edges with no vertex appearing twice

xA0 + xA1 + xA2 + xB0 + xB1 + xB5 + xC2 + xC3 + xC4
maximize
+ xD0 + xD1 + xE3 + xE4 + xE5 + xF2 + xF4 + xF5
Interpretation. Mutual preference constraints.
・People to jobs.
at most one job per person at most one person per job
xA0 + xA1 + xA2 ≤1 xA0 + xB0 + xD0 ≤1
・Students to writing seminars. xB0 + xB1 + xB5 ≤1 xA1 + xB1 + xD1 ≤1
subject
xC2 + xC3 + xC4 ≤1 xA2 + xC2 + xF2 ≤1
Alice Adobe to the
xD0 + xD1 ≤1 xC3 + xE3 ≤1
Adobe, Apple, Google Alice, Bob, Dave constraints
Bob Apple xE3 + xE4 + xE5 ≤1 xC4 + xE4 + xF4 ≤1
Adobe, Apple, Yahoo Alice, Bob, Dave A B C D E F
Carol Google xF2 + xF4 + xF5 ≤1 xB5 + xE5 + xF5 ≤1
Google, IBM, Sun Alice, Carol, Frank
Dave IBM
all xij ≥ 0
Adobe, Apple Carol, Eliza
Eliza Sun 0 1 2 3 4 5
IBM, Sun, Yahoo Carol, Eliza, Frank Theorem. [Birkhoff 1946, von Neumann 1953]
Frank Yahoo
Google, Sun, Yahoo Bob, Eliza, Frank All extreme points of the above polyhedron have integer (0 or 1) coordinates.
matching of cardinality 6: Corollary. Can solve matching problem by solving LP. not usually so lucky!
Example: job offers A–1, B–5, C–2, D–0, E–3, F–4
45 46

Linear programming perspective Universal problem-solving model (in theory)

Q. Got an optimization problem? Is there a universal problem-solving model?

Ex. Maxflow, bipartite matching, shortest paths, … [many, many, more] ・Maxflow.
・Shortest paths.
Approach 1: Use a specialized algorithm to solve it. ・Bipartite matching.
・Algorithms 4/e. ・Assignment problem. tractable

・Vast literature on algorithms. ・Multicommodity flow.

…
Approach 2: Use linear programming. ・Two-person zero-sum games.
・Many problems are easily modeled as LPs. ・Linear programming.
・Commercial solvers can solve those LPs. …
・Might be slower than specialized solution
(but you might not care). ・Factoring intractable ?

・NP-complete problems.
… see next lecture

Got an LP solver? Learn to use it!

Does P = NP? No universal problem-solving model exists unless P = NP.
47 48
 Algorithms R OBERT S EDGEWICK | K EVIN W AYNE

L INEAR P ROGRAMMING L INEAR P ROGRAMMING

‣ brewer’s problem ‣ brewer’s problem
‣ simplex algorithm ‣ simplex algorithm
‣ implementations ‣ implementations
Algorithms Algorithms
‣ reductions ‣ reductions
F O U R T H E D I T I O N

R OBERT S EDGEWICK | K EVIN W AYNE R OBERT S EDGEWICK | K EVIN W AYNE

http://algs4.cs.princeton.edu http://algs4.cs.princeton.edu