MIT OpenCourseWare

http://ocw.mit.edu

6.854J / 18.415J Advanced Algorithms

Fall 2008
��

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
1 18.415/6.854 Advanced Algorithms November 1994

Approximat ion Algorithms

Lecturer: Michel X . Goemans

Introduction
Many of the optimization problems we would like to solve are NP-hard. There are
several ways of coping with this apparent hardness. For most problems, there are
straightforward exhaustive search algorithms, and one could try to speed up such an
algorithm. Techniques which can be used include divide-and-conquer (or the refined
branch-and-bound which allows to eliminate part of the search tree by computing, at
every node, bounds on the optimum value), dynamic programming (which sometimes
leads to pseudo-polynomial algorithms), cutting plane algorithms (in which one tries
to refine a linear programming relaxation to better match the convex hull of integer
solutions), randomization, etc. Instead of trying to obtain an optimum solution, we
could also settle for a suboptimal solution. The latter approach refers to heuristic
or "rule of thumb" methods. The most widely used such methods involve some sort
of local search of the problem space, yielding a locally optimal solution. In fact,
heuristic methods can also be applied to polynomially solvable problems for which
existing algorithms are not "efficient" enough. A @(n1O)algorithm (or even a linear
time algorithm with a constant of 10lo0),although efficient from a complexity point
of view, will probably never get implemented because of its inherent inefficiency.
The drawback with heuristic algorithms is that it is difficult to compare them.
Which is better, which is worse? For this purpose, several kinds of analyses have
been introduced.
1. Empirical analysis. Here the heuristic is tested on a bunch of (hopefully
meaningful) instances, but there is no guarantee that the behavior of the heuris-
tic on these instances will be LLtypical"(what does it mean to be typical?).
2. Average-case analysis, dealing with the average-case behavior of a heuristic
over some distribution of instances. The difficulty with this approach is that it
can be difficult to find a distribution that matches the real-life data an algorithm
will face. Probabilistic analyses tend to be quite hard.
3. Worst-case analysis. Here, one tries to evaluate the performance of the
heuristic on the worst possible instance. Although this may be overly pes-
simistic, it gives a stronger guarantee about an algorithm's behavior. This is
the type of analysis we will be considering in these notes.
To this end, we introduce the following definition:

Approx- 1
Definition 1 The performance guarantee of a heuristic algorithm for a minimization
(maximization) problem is a if the algorithm is guaranteed to deliver a solution whose
value is at most (at least) a times the optimal value.
Definition 2 An a-approximation algorithm is a polynomial time algorithm with a
performance guarantee of a .
Before presenting techniques to design and analyze approximation algorithms as
well as specific approximation algorithms, we should first consider which performance
guarantees are unlikely to be achievable.

Negative Results
For some hard optimization problems, it is possible to show a limit on the performance
guarantee achievable in polynomial-time (assuming P # N P ) . A standard method
for proving results of this form is to show that the existence of an a-approximation
algorithm would allow you to solve some NP-complete decision problem in polynomial
time. Even though NP-complete problems have equivalent complexity when exact
solutions are desired, the reductions don't necessarily preserve approximability. The
class of NP-complete problems can be subdivided according to how well a problem
can be approximated.
As a first example, for the traveling salesman problem (given nonnegative lengths
on the edges of a complete graph, find a tour a closed walk visiting every vertex
-

exactly once - of minimum total length), there is no a-approximation for any a

unless P = NP. Indeed such an algorithm could be used to decide whether a graph
(V, E) has an Hamiltonian cycle (simply give every edge e a length of 1 and every
non-edge a very high or infinite length ).
As another example, consider the bin packing problem. You have an integer T
and weights X I , . . . ,x, E [0, TI, and you want to partition them into as few sets
("bins") as possible such that the sum of the weights in each set is at most T. It is
NP-complete to decide whether k bins are sufficient.
In fact, there is no a-approximation algorithm for this problem, for any a < 312.
To see this, consider the partition problem: given weights X I , .. . ,x, E [0,S] whose
total sum is 2S,is there a partition of the weights into two sets such that the sum in
each set is S? This is the same as asking: are two bins sufficient when each bin has
capacity S? If we had an a-approximation algorithm (a < 3/2), we could solve the
partition problem1. In general, if the problem of deciding whether a value is at most
k is NP-complete then there is no a-approximation algorithm with a < for the
problem of minimizing the value unless P = N P .
l ~ uwait,
t you exclaim -isn't there a polynomial-time approximation scheme for the bin packing
problem? In fact, very good approximation algorithms can be obtained for this problem if you allow
additive as well as multiplicative constants in your performance guarantee. It is our more restrictive
model that makes this negative result possible. See Section 6 for more details.
 Until 1992, pedestrian arguments such as this provided essentially the only known
examples of non-approximability results. Then came a string of papers culminating
in the result of Arora, Lund, Motwani, Sudan, and Szegedy[l] (based on the work of
Arora and Safra 121). They introduced a new characterization of N P in terms of prob-
abilistically checkable proofs (PCP). In the new characterization, for any language L
in NP, given the input x and a "proof" y of polynomial length in x, the verifier will
toss O(1og n) coins (where n is the size of x) to determine k = O(1) positions or bits
in the string y to probe; based on the values of these k bits, the verifier will answer
LLyes" or "no". The new characterization shows the existence of such a verifier V and
a proof y such that (i) if x E L then there exists a proof y such that V outputs LLyes"
independently of the random bits, (ii) if x f- L then for every proof y , V outputs "no"
with probability at least 0.5.
From this characterization, they deduce the following negative result for MAX
3SAT: given a set of clauses with at most 3 literals per clause, find an assignment
maximizing the number of satisfied clauses. They showed the following:

Theorem 1 For some E > 0, there is no 1 - E-approximation algorithm2 for MAX

3SAT unless P = NP.

The proof goes as follows. Take any NP-complete language L. Consider the
verifier V given by the characterization of Arora et al. The number of possible
output of the O(1og n) toin cosses is S = 2'('"gn) which is polynomial in n. Consider
any outcome of these coin tosses. This gives k bits, say i l , . . . , ik to examine in the
proof y. Based on these k bits, V will decide whether to answer yes or no. The
condition that it answers yes can be expressed as a boolean formula on these k bits
(with the Boolean variables being the bits of y). This formula can be expressed as
the disjunction ("or") of conjunctions ("and") of k literals, one for each satisfying
assignment. Equivalently, it can be written as the conjunction of disjunction of k
literals (one for each rejecting assignment). Since k is 0(1),this latter k-SAT formula
with at most 2k clauses can be expressed as a 3-SAT formula with a constant number
of clauses and variables (depending exponentially on k). (More precisely, using the
classical reduction from SAT to 3-SAT, we would get a 3-SAT formula with at most
k2k clauses and variables.) Call this constant number of clauses M 5 k2k = O(1).
If x E L, we know that there exists a proof y such that all SM clauses obtained by
concatenating the clauses for each random outcome is satisfiable. However, if x f- L,
for any y , the clauses corresponding to at least half the possible random outcomes
cannot be all satisfied. This means that if x 6 L, at least S/2 clauses cannot be
satisfied. Thus either all SM clauses can be satisfied or at most SM - $ clauses
can be satisfied. If we had an approximation algorithm with performance guarantee
better than 1 - E where E = & we could decide whether x E L or not, in polynomial

n definition of approximation algorithm, the performance guarantee is less than 1 for

2 ~ our
maxzmixatzon problems.
time (since our construction can be carried out in polynomial time). This proves the
theorem.
The above theorem implies a host of negative results, by considering the complex-
ity class MAX-SNP. defined by Papadimitriou and Yannakakis [21].

Corollary 2 For any MAX-SNP-complete problem, there is an absolute constant

6 > 0 such that there is no (1- 6)-approximation algorithm unless P = N P .

The class of MAX-SNP problems is defined in the next section and the corollary
is derived there. We first give some examples of problems that are complete for
MAX-SNP.

1. MAX 2-SAT: Given a set of clauses with one or two literals each, find an as-
signment that maximizes the number of satisfied clauses.

2. MAX k-SAT: Same as MAX 2-SAT, but each clause has up to k literals.

3. MAX CUT: Find a subset of the vertices of a graph that maximizes the number
of edges crossing the associated cut.

4. The Travelling Salesman Problem with the triangle inequality is MAX-SNP-

hard. (There is a technical snag here: MAX-SNP contains only maximization
problems, whereas TSP is a minimization problem.)

MAX-SNP Complete Problems

Let's consider an alternative definition of N P due to Fagin [9]. NP, instead of being
defined computationally, is defined as a set of predicates or functions on structures
G:

where $ is a quantifier free expression. Here S corresponds to the witness or the

proof.
Consider for example the problem SAT. We are given a set of clauses, where each
clause is the disjunction of literals. (A literal is a variable or its negation.) We want
to know if there is a way to set the variables true or false, such that every clause is
true. Thus here G is the set of clauses, S is the set of literals to be set to true, x
represents the clauses, y represents the literals, and

Where P(G, y, x ) is true iff y appears positively in clause x, and N(G, y , x ) is true iff
y appears negated in clause x.
 Strict NP is the set of problems in NP that can be defined without the the third
quantifier :

where $ is quantifier free.

An example is 3-SAT, the version of SAT in which every clause has at most 3
literals. Here x = (xl, x2,x3) (all possible combinations of three variables) and G is
the set of possible clauses; for example (xl V x2 V x3), (G V 5 2 V E ) ,and so forth.
Then $ is a huge conjunction of statements of the form: If (xl, $2, x3) appears as
(qV x2 V q),then x1 $ S V x2 E S V 5 3 $ S.
Instead of asking that for each x we get $(x, G, S), we can ask that the number
of x's for which $(x, G, S ) is true be maximized:

In this way, we can derive an optimization problem from an SNP predicate. These
maximization problems comprise the class MAX-SNP (MAXimization, Strict NP)
defined by Papadimitriou and Yannakakis 1211. Thus, MAX 3SAT is in MAX-SNP.
Papadimitriou and Yannakakis then introduce an L-reduction (L for linear), which
preserverses approximability. In particular, if P L-reduces to PI, and there exists an
a-approximation algorithm for P', then there exists a .ya-approximation algorithm
for P, where y is some constant depending on the reduction.
Given L-reductions, we can define MAX-SNP complete problems to be those P E
MAX-SNP for which Q S L P for all Q E MAX-SNP. Some examples of MAX-SNP
complete problems are MAX 3SAT, MAX 2SAT (and in fact MAX kSAT for any
fixed k > I ) , and MAX-CUT. The fact that MAX 3SAT is MAX-SNP-complete and
Theorem 1 implies the corollary mentioned previously.
For MAX 3SAT, E in the statement of Theorem 1 can be chosen can be set to
1/ 74 (Bellare and Sudan [5]).
Minimization problems may not be able to be expressed so that they are in MAX-
SNP, but they can still be MAX-SNP hard. Examples of such problems are:

TSP with edge weights 1 and 2 (i.e., d(i, j) E {1,2} for all i, j ) . In this case,
there exists a 716-approximation algorithm due to Papadimitriou and Yan-
nakakis.

Steiner tree with edge weights 1 and 2.

Minimum Vertex Cover. (Given a graph G = (V, E), a vertex cover is a set
S & V such that (u,v) E E j u E S or v E S.)
 The Design of Approximation Algorithms
We now look at key ideas in the design and analysis of approximation algorithms.
We will concentrate on minimization problems, but the ideas apply equally well to
maximization problems. Since we are interested in the minimization case, we know
that an a-approximation algorithm H has cost CH 5 aCOpTwhere COPTis the cost
of the optimal solution, and a 2 1.
Relating CH to COPTdirectly can be difficult. One reason is that for NP-hard
problems, the optimum solution is not well characterized. So instead we can relate
the two in two steps:

Here L B is a lower bound on the optimal solution.

Relating to Optimum Directly

This is not always necessary, however. One algorithm whose solution is easy to relate
directly to the optimal solution is Christofides' [6] algorithm for the TSP with the
+
triangle inequality (d(i,j) d(j, k ) 5 d(i, k ) for all i, j, k). This is a :-approximation
algorithm, and is the best known for this problem. The algorithm is as follows:

1. Compute the minimum spanning tree T of the graph G = (V,E ) .

2. Let 0 be the odd degree vertices in T. One can prove that 101 is
' even.

3. Compute a minimum cost perfect matching M on the graph induced by 0.

4. Add the edges in M to E. Now the degree of every vertex of G is even. Therefore
G has an Eulerian tour. Trace the tour, and take shortcuts when the same vertex
is reached twice. This cannot increase the cost since the triangle inequality
holds.

We claim that Zc 5 :ZTSP, where Zc is the cost of the tour produced by

Christofides' algorithm, and ZTsP is the cost of the optimal solution. The proof
is easy:

Here ZT is the cost of the minimum spanning tree and ZM is the cost of the matching.
Clearly ZT 5 ZTSP,since if we delete an edge of the optimal tour a spanning tree
results, and the cost of the minimum spanning tree is at most the cost of that tree.
Therefore -$& 5 1.
 To show & $ 5 consider the optimal tour visiting only the vertices in 0.
Clearly by the triangle inequality this is of length no more than ZTSP. There are an
even number of vertices in this tour, and so also an even number of edges, and the
tour defines two disjoint matchings on the graph induced by 0. At least one of these
has cost 5 $ZTSP, and the cost of ZM is no more than this.

3.2 Using Lower Bounds

Let

A lower bound on COPTcan be obtained by a so-called relaxation. Consider a related

optimization problem L B = minxERg(x). Then L B is a lower bound on COPT(and
the optimization problem is called a relaxation of the original problem) if the following
conditions hold:

(2) g(x) 5 f (x) for all x E S.

Indeed these conditions imply

LB = ming(x) 5 min f ( x ) = COPT.

XER XES

Most classical relaxations are obtained by using linear programming. However,

there are limitations as to how good an approximation LP can produce. We next
show how to use a linear programming relaxation to get a 2-approximation algorithm
for Vertex Cover, and show that this particular L P relaxation cannot give a better
approximation algorithm.

An L P Relaxation for Minimum Weight Vertex Cover

A vertex cover U in a graph G = (V, E) is a subset of vertices such that every edge is
incident to at least one vertex in U . The vertex cover problem is defined as follows:
Given a graph G = (V, E) and weight w(v) 2 0 for each vertex v, find a vertex cover
U V minimizing w(U) = w(v). (Note that the problem in which nonpositive
weight vertices are allowed can be handled by including all such vertices in the cover,
deleting them and the incident edges, and finding a minimum weight cover of the
remaining graph. Although this reduction preserves optimality, it does not maintain
approximability; consider, for example, the case in which the optimum vertex cover
has 0 cost (or even negative cost) .)
 This can be expressed as an integer program as follows. Let x(v) = 1 if v E U
and x(v) = 0 otherwise. Then

COPT= min C W ( V ) X ( V )
x E S vEV

where

We now relax S, turning the problem into a linear program:

L B = min w(v)x(v)
xER V E V

In order to show that R is a relaxation, we must show that it satisfies conditions 1

and 2. Condition 1 clearly holds, as 0 , l 2 0. Furthermore, condition 2 also holds,
since the objective function is unchanged. Thus, we can conclude that L B 5 COPT,
and we can prove that an algorithm H is an a-approximation algorithm for VC by
showing CH 5 a L B .
The limitation of this relaxation is that there are instances where L BN +COPT.
This implies that it cannot be used to show any a < 2, since if we could then
H would give a better answer than the optimum. One such instance is K,: the
complete graph on n vertices, with all vertices weight 1. All the nodes but one must
be in the cover (otherwise there will be an edge between two that are not, with neither
in the cover set). Thus, COPT= n - 1. The relaxation, on the other hand, can have
x(v) = f ,YV E V. Thus, L B 5 ,; which means L B f COPT.

How to use Relaxations

There are two main techniques to derive an approximately optimal solution from a
solution to the relaxed problem.

1. Rounding
Find an optimal solution x* to the relaxation. Round x* E R to an element
x' E S. Then prove f (x') 5 ag(x*) which implies

Often randomization is helpful, as we shall see in later sections. In this case

x* E R is randomly rounded to some element x' E S so that E [f(x')] 5 ag(x*).
These algorithms can sometimes be derandomized, in which case one finds an
x" such that f(xN)5 E[f(x')].
 2. Primal-Dual
Consider some weak dual of the relaxation:

Construct x E S from y E D such that

Notice that y can be any element of D, not necessarily an optimal solution to

the dual.
We now illustrate these techniques on the minimum weight vertex cover problem.

3.4.1 Rounding applied to VC

This is due to Hochbaum [16]. Let x* be the optimal solution of the LP relaxation.
Let

We claim U is a 2-approximation of the minimum weight VC. Clearly U is a vertex

+
cover, because for (u, v) E E we have x*(u) x*(v) 2 1, which implies x*(u) 2 112
or x*(v) 2 112. Also
x W(V) 5
vEU
xw ( v ) ~ x * ( v=
vEV
) 2LB

since 2x*(v) 2 1 for a11 v E U .

3.4.2 Primal-Dual applied to VC

This is due to Bar-Yehuda and Even [4]. First formulate the dual problem. Let
1 ; elements of y are y(e) for e = (u, v) E E. The dual is:
y E ~ 1 ~ the

Initialize C (the vertex cover) to the empty set, y = 0 and F = E. The algorithm
proceeds by repeating the following two steps while F # 0:
1. Choose some e = (u, v) E F. Increase y (e) as much as possible, until inequal-
ity (3) becomes tight for u or v. Assume WLOG it is tight for u.
2. Add u to C and remove all edges incident to u from F.
Clearly C is a vertex cover. Furthermore
 The Min-Cost Perfect Matching Problem
In this section, we illustrate the power of the primal-dual technique to derive approx-
imat ion algorithms. We consider the following problem.

Definition 3 The Minimum-Cost Perfect Matching Problem (MCPMP) is as fol-

lows: Given a complete graph G = (V, E) with IVI even and a nonnegative cost
function c, 2 0 on the edges e E E, find a perfect matching M such that the cost
c(M) is minimized, where c(M) = CeEM
Ce.

The first polynomial time algorithm for this problem was given by Edmonds [8] and
has a running time of O(n4) where n = IVI. To date, the fastest strongly polynomial
time algorithm is due to Gabow [lo] and has a running time of O(n(m n 1g n)) +
where m = IEl. For dense graphs, m = O(n2),this algorithm gives a running time
of O(n3). The best weakly polynomial algorithm is due to Gabow and Tarjan [12]
and runs in time 0(md-lognC) where C is a bound on the costs c,.
For dense graphs with C = O(n), this bound gives an O*(n2-5) running time.
As you might suspect from these bounds, the algorithms involved are fairly com-
plicated. Also, these algorithms are too slow for many of the instances of the problem
that arise in practice. In this section, we discuss an approximation algorithm by Goe-
mans and Williamson 1131 that runs in time O(n2lgn). (This bound has recently
been improved by Gabow, Goemans and Williamson [ l l ] to O(n(n J-)).) +
Although MCPMP itself is in PTIME, this algorithm is sufficiently general to give
approximations for many NP-hard problems as well.
The algorithm of Goemans and Williamson is a 2-approximation algorithm - it
outputs a perfect matching with cost not more than a factor of 2 larger than the cost
of a minimum-cost perfect matching. This algorithm requires that the costs c, make
+
up a metric, that is, c, must respect the triangle inequality: cij cjk 2 cik for all
triples i , j , k of vertices.

A linear programming formulation

The basic idea used in the 2-approximation algorithm of Goemans and Williamson
is linear programming and duality. The min-cost perfect matching problem can be
formulated as a linear program. The algorithm does not directly solve the linear
program, but during its operation, it can compute a feasible solution to the dual
program. This dual feasible solution actually certifies the factor of 2 approximation.
Before writing down the linear program, we start with an observation.
Consider a matching M and a set S c V of vertices with IS1 odd. If M is a perfect
matching, then since IS1 is odd, there must be some edge in the matching that has
one endpoint inside S and the other outside. In other symbols, let 6(S) be the set of
edges in E with exactly one endpoint in S; if M is a perfect matching and IS1 is odd,
then M n 6(S) # @.
 With this observation, we can now formulate MCPMP as a linear program:

Z = M i n x cex,
eEE
subject to: x xe 2 1
~EG(S)
for all S c V with IS1 odd

for all e E E.

We can now see that the value Z of this linear program is a lower bound on the cost
of any perfect matching. In particular, for any perfect matching M, we let

1 if e E M;
0 otherwise.
Clearly, this assignment is a feasible solution to the linear program, so we know that
Z 5 c(M). This bound also applies to a minimum-cost perfect matching M*, so we
have Z I c(M*).
Note that this is a huge linear program having one constraint for each S c V of
odd cardinality. Though it is too large to be solved in polynomial time by any of the
linear programming algorithms we have seen, the ellipsoid method can actually solve
this program in polynomial time. We do not consider this solution technique; rather
we let the linear program and its dual serve as a tool for developing and analyzing
the algorithm.
We now consider the dual linear program:

Z=Max x
scv,
ys
IS1 odd

subject to: for all e E E

for all S c V with IS1 odd.

Note that by strong duality, the value Z of this dual linear program is the same as
the value Z of the primal program.
This dual linear program is used to verify that the perfect matching output by
the algorithm is actually within a factor of 2 of optimal. The algorithm outputs two
things:
1. a perfect matching MI, and

2. a dual feasible solution y such that

c(Mt) I 2 C Ys.
scv,
IS1 odd

Approx- 11
Since y is dual feasible, we know that

YS 5 5 c(M)
scv,
IS1 odd

where M is any perfect matching. Thus we have

where M* is a min-cost perfect matching. The algorithm is therefore (given that it

runs in polynomial time) a 2-approximation algorithm for MCPMP.
To be precise, the algorithm need not actually output the dual feasible solution
y - it is only needed as an analysis tool to prove the factor of 2 approximation
bound. In spite of the fact that there are an exponential number of y s variables, the
algorithm could actually compute and output the y s values since it turns out that
only a polynomial number of them are non-zero. When we finally get to exhibiting
the algorithm, we will include the computation of the y s values.

From forest to perfect matching

Rather than directly compute the perfect matching M', the algorithm first computes
a forest F' from which M' can be derived. In the forest F', all components have
even size, and furthermore, F' is edge-minimal in the sense that if any edge of F' is
removed, then the resulting forest has an odd size component. Additionally, the cost
of F' is bounded by twice the value of the dual feasible solution; that is,

c(F') 52 C Ys.
sc v,
IS1 odd

We now show how to convert F' into a perfect matching M' such that c(Mr) 5
c(Fr). The idea is as follows. Starting from the forest F', consider any vertex v with
degree at least 3. Take two edges (u, v) and (v, w); remove them and replace them
with the single edge (u, w). Since the edge costs obey the triangle inequality, the
resulting forest must have a cost not more than c(F'). Thus, if we can iterate on this
operation until all vertices have degree 1, then we have our perfect matching M'.
The only thing that can get in the way of the operation just described is a vertex
of degree 2. Fortunately, we can show that all vertices of F' have odd degree. Notice
then that this property is preserved by the basic operation we are using. (As a direct
consequence, the property that all components are even is also preserved.) Therefore,
if all vertices of F' have odd degree, we can iteration the basic operation to produce
a perfect matching M' such that c(M') 5 c(Fr). Notice that M' is produced after
O ( n ) iterations.

Lemma 3 All vertices of F' h a v e odd degree.

Approx- 12
Proof: Suppose there is a vertex v with even degree, and let v be in component
A of F'. Removing v and all its incident edges partitions A into an even number k of
smaller components Al, A2, . . . ,Ak. If all k of these components have odd size, then
it must be the case that A has odd size. But we know that A has even size -- all
components of F' have even size -- so there must be a component Ai with even size.
Let vi denote the vertex in Ai such that (v,vi) is an edge of F'. Now if we start from
F' and remove the edge (v, vi), we separate A into two even size components. This
contradicts the edge-minimality of F'.

The algorithm
The algorithm must now output an edge-minimalforest F' with even size components
and be able to compute a dual feasible solution y such that c ( F f )I2 C ys.
At the highest level, the algorithm is:
1. Start with F = 0.

2. As long as there exists an odd size component of F, add an edge between two
components (at least one of which has odd size).
Note that the set of components of F is initially just the set of vertices V.
The choice of edges is guided by the dual linear program shown earlier. We start
with all the dual variables equal to zero; y s = 0. Suppose at some point in the
execution we have a forest F as shown below and a dual solution y. Look at the

components S of odd cardinality (components 1, 4, 6 and 7, in this case). For these

components, increase y s by some 6 , leaving all other values of y s unchanged. That
is,

Ys+
lYs+ 6 if S is an odd size component of F
, 7

otherwise.
Make 6 as large as possible while keeping y s dual feasible. By doing this, we make
the constraint on some edge e tight; for some e the constraint

becomes
This is the edge e that we add to F . (If more than one edge constraint becomes tight
simultaneously, then just arbitrarily pick one of the edges to add.)
We now state the algorithm to compute F'. The steps that compute the dual
feasible solution y are commented out by placing the text in curly braces.

F +0
C t {{i)I i E V ) {The components of F )
{Let ys t 0 for all S with IS1 odd.)
Vi E V do d(i) + 0 { d ( i )= Cs3iys)
while 3C E C with ICI odd do
Find edge e = ( i ,j ) such that i E C,, j E C,, p f q
ce-d(i)-d(j)
which minimizes 6 =
~(CP)+~(C,)
where X(C) = { ' / ~ ~ ~ (i.e., s ~ parity of C ) .
w ~ the

F -+F U { e )
VC E C with ICI odd do
Yi E C do d(i) t d(i)+ 6
{Let Yc + Yc 6.) +
c c \ {CP,C,) u {C, u C,)
+

F' t edge-minimal F

Analysis of the algorithm

Lemma 4 The values of the variables ys computed b y the above algorithm constitute
a dual feasible solution.

Proof: We show this by induction on the while loop. Specifically, we show that
at the start of each iteration, the values of the variables ys are feasible for the dual
linear program. We want to show that for each edge e E E,

scv,
e€S(S)

The base case is trivial since all variables ys are initialized to zero and the cost
function c, is nonnegative. Now consider an edge e' = ( i f j') , and an iteration. There
are two cases to consider.
In the first case, suppose both 'i and j' are in the same component at the start
of the iteration. In this case, there is no component C E C for which e' E S(C).
Therefore, since the only way a variable ys gets increased is when S is a component,
none of the variables ys with e' E 6 ( S )get increased at this iteration. By the induction
hypothesis, we assume the iteration starts with

scv,
e1€S(S)

and therefore, since the left-hand-side of this inequality does not change during the
iteration, this inequality is also satisfied at the start of the next iteration.
In the second case, suppose i' and j' are in different components; i' E Cpl and
j' E Cql at the start of the iteration. In this case, we can write

scv,
CYS+ scv,
C Ys = scv, Cys

where d(i) is defined by

d(i) = C y s .
scv,
iES
The equality follows because since i1and j' are in different components, if S contains
both 'i and j', then S is not and never has been a component; hence, for such a set
S, we have ys = 0. We know that during this iteration d(il)will be incremented by 6
if and only if ycp1is incremented by 6, and this occurs if and only if X(C,t) = 1. Let
d'(il)and d ' ( j t ) be the new values of d(il)and d ( j l ) after this iteration. Then we have
d1(i')= d(il)+ 6X(Cp!), and
d t ( j t ) = d ( j l ) + SX(C,l).
Now, by the way 6 is chosen, we know that

Thus, at the beginning of the next iteration we have

Finally, for completeness sake, we note that the constraint ys 2 0 is satisfied

because ys = 0 initially and 6 0. >
Approx- 15
 As a final observation, we note that when the algorithm selects an edge e', the
corresponding constraint in the dual linear program becomes tight. This means that
for all edges e E F , we have
C Ys = ce*
SCV*
e€G(S)

A simulated run of the algorithm

Since the algorithm as given can be difficult to visualize, here is an example of how
it would execute. See Figure 1.

Figure 1: A sample run of the algorithm. The various values of d ( i ) are indicated b y
the shaded regions around the components.

We'll assume a Euclidean distance metric to ease visualization. Now, initially, all
points (1through 8 ) are in separate components, and d ( i ) is 0 for all i. Since the metric
is Euclidean distance, the first edge to be found will be (7,8). Since both components
are of odd size, 6 will be half the distance between them ( ( c , - 0 - 0 ) / ( 1 + 1 ) ) . Since,
in fact, all components are of odd size, every d(i) will be increased by this amount,
as indicated by the innermost white circle around each point. The set { 7 , 8 ) now
becomes a single component of even size.
In general, we can see the next edge to be chosen by finding the pair of components
whose boundaries in the picture can be most easily made to touch. Thus, the next
edge is (3,5), since the boundaries of their regions are closest, and the resulting values

Approx-16
of d ( i ) are represented by the dark gray bands around points 1 through 6. Note that
the component {7,8} does not increase its d ( i ) values since it is of even size.
We continue in this way, expanding the "moats" around odd-sized components
until all components are of even size. Since there is an even number of vertices and
we always expand odd-sized components, we are guaranteed to reach such a point.

Final Steps of Algorithm and Proof of Correctness

Let F' = { e E F : F \ { e ) has an odd sized component}. We will show that the F' so
obtained is a forest with components of even cardinality and that it is edge minimal
with respect to this property. It is obvious that F' is a forest since F' c F and F is
a forest. We will also show that the cost of this forest ( F ' ) , is less than or equal to
twice the dual solution. In section 4.2 we showed how to build a matching from this
forest with the cost of the matching less than or equal to the cost of the forest. Thus,
this gives us a 2-approximation algorithm for matching. As an example see the figure
given below.

Figure 2: Example showing how to get F' from F .

Theorem 5 Let F' = { e E F : F \ { e ) has an odd sized component). Then

1. every component of F' has an even number of vertices and F' is edge minimal
with respect to this property..

Proof:
Let us first show that every component of F' has an even number of vertices.
Suppose not. Then consider the components of F . Every component of F has an
even number of vertices by design of the algorithm. Consider a component of F' which
has an odd number of vertices and let us denote it as T:. Let Tibe the component
that T,! belongs to in F . Let Nl, . . . ,Ni be the components of F within Ti obtained
by removing T: (see figure 3). Ti has an even number of vertices. Nk with 1 5 k 5 j
has an even number of vertices because, otherwise, the edge from Nk to T;I would
belong to F' by definition. But this implies that T,! is even, which is a contradiction.

Figure 3: Every component of F' has an even # of vertices.

A simple proof by contradiction shows that F' is edge minimal. Suppose F' is not
edge minimal. Then there is an edge or set of edges which can be removed which leave
even sized components. Consider one such edge e. It falls into one of two categories:
1. Its removal divides a component into two even sized components. But this
means that e was already removed by the definition of F'.

2. Its removal divides a component into two odd sized components. Despite the
fact that other edges may be removed, as well, an two odd sized component will
remain in the forest. Thus, e cannot be removed.
Now let us prove the second portion of the theorem. In what follows, though we
do not explicitly notate it, when we refer to a set S of vertices, we mean a set S of
vertices with I
S1 odd. We observe that by the choice of the edges e in F, we have

for all e E F . Thus,

Thus we need to show,

 We will show this by induction. In what follows, bear in mind that F' is what
we have, at the end of the algorithm. We will show the above relation holds at every
iteration.
Initially, ys = 0. Thus the LHS and RHS are both zero. Thus, this is true initially.
Let us suppose it is true at any intermediate stage in the algorithm. We will show
that it will remain true in the next iteration. From one iteration to the next the only
ys that change are those with C E C with ICI odd. Thus if we show the increase in
the LHS is less than the RHS we are done. i.e.

6 C I~'n6(~)l~26l{C~C,lclodd)l
c~c,lclodd

x odd
c~c,lcl
IF'nS(C)II21{CEC,ICIodd]l

Now, define a graph H with C E C as vertices, with an edge between Cp and C, if

there exists an edge in F' n {6(Cp)n 6(Cq)). We can partition these vertices into two
groups based on their cardinality. Those that have even cardinality and those that
have odd cardinality. Remove from this graph all vertices that have even cardinality
and are isolated (they have no edges incident to them). We will denote the resulting
set of vertices of odd and even cardinality by Odd and Even respectively.
C c ~ c , odd
~c~I F' n 6(C)l corresponds to the sum of the degrees of vertices in
Odd in the graph H. And, I{C E C, ICI odd)l, corresponds to the number of vertices
in odd. Thus we need to show:

where dH(v) denotes the degree of node v in the graph H. Since F' is a forest, H is
also a forest and we have:
Number of edges in H 5 number of vertices in H. Or

We now claim that if v E Even then v is not a leaf. If this is true then (2-dH(v)) 5
0 for v E Even and so we are done.
Suppose there is a vi E Even which is a leaf. Consider the component C in H
that vi is contained in. By the construction of H , each tree in F' is either contained
solely in the vertices represented by C or it is strictly outside C. Since each tree in
F' contains an even number of vertices C does (w.r.t. the original graph), as well.
So vi and C - vi each contains an even number of vertices. As a result, removing the
edge between vi and G - vi would leave even sized components, thus contradicting
the minimality of F'.

Figure 4: dH(v) 2 2 for v E Even

Some implementation details

The algorithm can be implemented in O(n2logn). For this purpose, notice that
the number of iterations is at most n - 1 since F is a forest. The components
of F can be maintained as a union-find structure and, therefore, all mergings of
components take O(na(n,n)) where a is the inverse Ackermann function. In order
to get the O(n2log n) bound, we shall show that every iteration can be implemented
in O(n log n) time.
In order to find the edge e to add to F, we maintain a priority queue containing
the edges between different components of F. This initialization of this priority queue
takes 0 ( n 2log n) time. In order to describe the key of an edge, we need to introduce
a notion of time. The time is initially set to zero and increases by 6 in each iteration
(the time can be seen to be the maximum of di over all vertices i). The key of an
+
edge e = (i, j ) is equal to the time at which we would have c, = di dj if the
parity of the components containing i and j don't change. The edge to be selected
is therefore the edge with minimum key and can be obtained in O(n log n). When
two components merge, we need to update the keys of edges incident to the resulting
component (since the parity might have changed). By keeping track of only one edge
between two components (the one with smallest key), we need to update the keys of
O(n) edges when two components merge. This can be done in O(n log n) (O(1ogn)
per update).
To complete the discussion, we need to show how to go from F to F'. By per-
forming a post-order traversal of the tree and computing the parity of the subtrees
encountered (in a recursive manner), this step can be implemented in O(n) time.
 Approximating MAX-CUT
In this section, we illustrate the fact that improved approximation algorithms can be
obtained by considering relaxations more sophisticated than linear ones. At the same
time, we will also illustrate the fact that rounding a solution from the relaxation in a
randomized fashion can be very useful. For this purpose, we consider approximation
algorithms for the MAX-CUT problem. The unweighted version of this problem is as
follows:
Given: A graph G = (V, E).
Find: A partition (S,S) such that d(S) := IS(S)I is maximized.
It can be shown that this problem is NP-hard and MAX SNP-complete and so
we cannot hope for an approximation algorithm with guarantee arbitrarily close to I
unless P = NP. In the weighted version of the problem each edge has a weight wij
and we define d(S) by,
4s)= C
(6,3)E E : ~ E j$S
S,

For simplicity we focus on the unweighted case. The results that we shall obtain will
also apply to the weighted case.
Recall that an a-approximation algorithm for MAX- CUT is a polynomial time
algorithm which delivers a cut 6(S) such that d(S) 2 a z ~ where
c ZMC is the value
of the optimum cut. Until 1993 the best known a was 0.5 but now it is 0.878 due
to an approximation algorithm of Goemans and Williamson [14]. We shall first of all
look at three (almost identical) algorithms which have an approximation ratio of 0.5.
1. Randomized construction. We select S uniformly from all subsets of V. i.e.
i
For each i E V we put i E S with probability (independently of J' # i).

E [d(S)I = z ( i , j ) E Pr
~ [(i,j) E b(S)1 by linearity of expect ations
= C ( i , j ) E E P r [ i ~ S , j $ S O r i $ S]
S,j~
= iIEl.

But clearly ZMC 5 IEl and so we have E [d(S)]2 $zMc. Note that by comparing
our cut to IEl, the best possible bound that we could obtain is $- since for K,
(the complete graph on n vertices) we have IE 1 = (:) and z ~ =c7 n2
.

2. Greedy procedure. Let V = {1,2,. . . ,n) and let E j = {i : (i,j ) E E and i <
j}. It is clear that {Ej : j = 2, . . . ,n} forms a partition of E. The algorithm
is:

Set S = (1)
For j = 2 t o n do
if lSnEjI 5 ~ I E ~ I
then S + S U {j).
 If we define Fj = Ej n 6 ( S ) then we can see that {Fj : j = 2,. .. , n ) is a
partition of S(S). By definition of the algorithm it is clear that IFj 2 I
By 9.
summing over j we get d ( S ) 2 2 y.In fact, the greedy algorithm can
be obtained from the randomized algorithm by using the method of conditional
expectations.

3. Local search. Say that 6 ( S )is locally optimum if 'di E S : d(S - { i ) ) 5 d ( S )

and 'di $ S : d(S U { i ) ) 5 d ( S ) .

Lemma 6 if S(S) is locally optimum then d ( S ) 2 F.

Proof:

d(S) = A x i n u m b e r of edges in cut incident to i )

iEv

The inequality is true because if IS(S) n S(i)1 < IS(i)1 for some i then we can
move i to the other side of the cut and get an improvement. This contradicts
local optimality.
In local search we move one vertex at a time from one side of the cut to the other
until we reach a local optimum. In the unweighted case this is a polynomial
time algorithm since the number of different values that a cut can take is O(n2).
In the weighted case the running time can be exponential. Haken and Luby [15]
have shown that this can be true even for 4-regular graphs. For cubic graphs
the running time is polynomial [22].

Over the last 15-20 years a number of small improvements were made in the approxi-
mation ratio obtainable for MAX-CUT. The ratio increased in the following manner:

where m = IEI and n = IVI, but asymptotically this is still 0.5.

 Randomized 0.878 Algorithm
The algorithm that we now present is randomized but it differs from our previous
randomized algorithm in two important ways.

a The event i E S is not independent from the event j E S.

We compare the cut that we obtain to an upper bound which is better that IEl.

Figure 5 : The sphere S,.

Suppose that for each vertex i E V we have a vector vi E Rn (where n = IVI). Let
Sn be the unit sphere {x E Rn : 1 lx 1 1 = 1). Take a point r uniformly distributed on
Sn and let S = {i E V : vi r 2 0) (Figure 5). (Note that without loss of generality
llvill = 1.) Then by linearity of expectations:

E [d(S)]= C P r [sign(vi r ) f sign(vj r)].

EE

Lemma 7

P r [sign(vi r) # sign(vj . r)] = Pr [random hyperplane separates vi and vj]

a
- -
-
7T

-
where a = arccos(vi v j ) (the angle between vi and vj).

Proof: Thisresultiseasytoseebutitisalittledifficulttoformalize.LetPbethe
2-dimensional plane containing vi and vj. Then P n Sn is a circle. With probability
1, H = {x : x * r = 0) intersects this circle in exactly two points s and t (which are
diametrically opposed). See figure 6. By symmetry s and t are uniformly distributed
on the circle. The vectors vi and v j are separated by the hyperplane H if and only if
either 3 or t lies on the smaller arc between vi and vj. This happens with probability
2a -
- 2
27r - 7r*
 Figure 6: The plane P.

From equation 5 and lemma 7 we obtain:

Observe that E [d(S)]5 z ~ and

c so

where we maximize over all choices for the vi7s. We actually have ma%, E [d(S)]=
ZMC Let S(T)be a cut such that d ( T )= z ~ and
c let e be the unit vector whose first
component is 1 and whose other components are 0. If we set

e i f i ~ T
Vi =
-e otherwise.

then S(S)= S(T) with probability 1. This means that E [d(S)]= z ~ c

Corollary 8

Unfortunately this is as difficult to solve as the original problem and so at first glance
we have not made any progress.
 Choosing a good set of vectors
Let f : [- 1,1] + [0,1] be a function which satisfies f (- 1) = 1, f (1) = 0. Consider
the following program:

Max

subject to:

If we denote the optimal value of this program by zp then we have z ~ 5c zp. This
is because if we have a cut 6(T) then we can let,

vi = { e
-e
ifiET
otherwise.

Hence C(i,j)EE
f (vi vj) = d(T) and z ~ 5c zp follows immediately.

5.3 The Algorithm

The framework for the 0.878 approximation algorithm for MAX-CUT can now be
presented.

1. Solve (P) to get a set of vectors {v;, .. . ,v:).

2. Uniformly select r E S,.

3. Set S = {i : v b - r > 0).

Theorem 9

where,
arccos(x)
a = min
-15.51 ?rf(x) '

Proof:

(i,j)€E
= azp
 We must now choose f such that (P) can be solved in polynomial time and a is as
large as possible. We shall show that (P)can be solved in polynomial time whenever
f is linear and so if we define,
1-x
f (4= - 2
then our first criterion is satisfied. (Note that f (-1) = 1 and f (1) = 0.) With this
choice of f ,
2 arccos(x)
a = min
-15x51 T(1 - x)

(See figure 7.)

Figure 7: Calculating a.

Solving ( P )
We now turn our attention to solving:

Man C l v i - v j )
(i,j)€E
subject to:
Let Y = (yij) where yij = vi vj. Then,

llvill = 1 + yii = 1 for all i.

yij = vi vj +=
Y k 0, where Y k 0 means that Y is positive semi-definite:
Vx : xTYx 2 0.) This is true because,

Conversely if Y k 0 and yii = 1 for all i then it can be shown that there exists a set
of vi's such that yij = vi vj. Hence (P) is equivalent to,

Max

subject to:

> >
Note that Q := { Y : Y 0,yii = 1) is convex. (If A 0 and B 0 then A + B 0 >
and also > 0.) It can be shown that maximizing a concave function over a
convex set can be done in polynomial time. Hence we can solve ( P I ) in polynomial
time since linear functions are concave. This completes the analysis of the algorithm.

5.5 Remarks
1. The optimum Y could be irrational but in this case we can find a solution with
an arbitrarily small error in polynomial time.

2. To solve (P') in polynomial time we could use a variation of the interior point
method for linear programming.

3. Given Y, vi can be obtained using a Cholesky factorization (Y = VVT).

4. The algorithm can be derandomized using the method of conditional expecta-

tions. This is quite intricate.

5. The analysis is very nearly tight. For the 5-cycle we have z ~ and
c zp =
+
(1 cos = t) which implies that 2P = 0.88445.
 Bin Packing and P 11 Cmax
One can push the notion of approximation algorithms a bit further than we have been
doing and define the notion of approximation schemes:

Definition 4 A polynomial approximation scheme (pas) is a family of algorithms

+
A, : t > O such that for each t > 0, A, is a (1 t)-approximation algorithm which
runs in polynomial time in input size for fixed t.

Definition 5 A fully polynomial approximation scheme (fpas) is a pas with running

time which is polynomial both in input size and l/t.

It is known that if n is a strongly NP-complete problem, then n has no fpas unless

P = N P . From the result of Arora et al. described in Section 2, we also know that
there is no pas for any M A X - SNP hard problem unless P = N P .
We now consider two problems which have a very similar flavor; in fact, they
correspond to the same NP-complete decision problem. However, they considerably
differ in terms of approximability: one has a pas, the other does not.

a B i n Packing: Given item sizes a l , az,. . . ,a, 2 0 and a bin size of T, find a
partition of 11,. . . ,Ikof 1 , . . . ,n, such that CiEI,ai 5 T and k is minimized
(the items in Ii are assigned to bin 1).

a P 11 C,,,: Given n jobs with processing times pl, . . . ,p, and m machines,
find a partition {Il, . . . ,I,) of (1, . . . ,n}, such that the makespan defined as
maxi (CjEI, pj ) is minimum. (The makespan represents the maximum comple-
tion time on any machine given that the jobs in I, are assigned to machine
i) .

The decision versions of the two problems are identical and NP-complete. However
when we consider approximation algorithms for the two problems we have completely
different results. In the case of the bin packing problem there is no a-approximation
algorithm with a < 312, unless P = NP.

Proposition 10 There is no a-approximation algorithm with a < 312, for bin pack-
ing, unless P = NP, as seen in Section 2.

However, we shall see, for P 11 C,,, we have a-approximation algorithms for any

Definition 6 An algorithm A has an asymptotic performance guarantee of a if

a 2 lim sup a k
k+m
where
at;:= sup \ I

I:OPT(I)=k O P T ( I )

a n d O P T ( I ) denotes t h e value of instance I a n d A(I) denotes t h e value returned b y

algorithm A.

11
For P Cmax,there is no difference between an asymptotic performance and a per-
formance guarantee. This follows from the fact that P 11 Cmaxsatisfies a scaling
property : an instance with value P O P T ( I ) can be constructed by multiplying every
processing time pj by P.
Using this definition we can analogously define a polynomial a s y m p t o t i c approx-
i m a t i o n s c h e m e (paas). And a fully polynomial a s y m p t o t i c approximation s c h e m e
(fpaas).
Now we will state some results to illustrate the difference in the two problems
when we consider approximat ion algorithms.

1. For bin packing, there does not exist an a-approximation algorithm with a <
312, unless P = N P . Therefore there is no pas for bin packing unless P = N P .

11
2. For P Cmaxthere exists a pas. This is due to Hochbaum and Shmoys [17].
We will study this algorithm in more detail in today's lecture.

3. For bin packing there exists a paas. (Fernandez de la Vega and Lueker [7]).

4. For P 11 Cmaxthere exists no fpaas unless P = N P . This is because the existence

of a fpaas implies the existence of a fpas and the existence of a fpas is ruled out
unless P = N P because, P 11 Cmaxis strongly NP-complete.

5. For bin packing there even exists a fpaas. This was shown by Karmarkar and
Karp [la].

Approximat ion algorithm for PI [C,,,

We will now present a polynomial approximation scheme for the P Cmaxscheduling II
problem.
We analyze a pas for PI ICmax,discovered by Hochbaum and Shmoys [17]. The
idea is to use a relation similar to the one between an optimization problem and
its decision problem. That is, if we have a way to solve decision problem, we can
use binary search to find the exact solution. Similarly, in order to obtain a (1 6 ) - +
+
approximation algorithm, we are going to use a so-called (1 €)-relaxed decision
version of the problem and binary search.

Definition 7 (1 + 6) -relaxed decision version of PI I Cmaxi s a procedure t h a t given t

a n d a deadline T, returns either:
 1 I I
2 Pj max pi

Figure 8: List scheduling.

"NO" - if there does not exist a schedule with makespan 5 T , or

"YES" - if a schedule with makespan 5 ( 1 + c)T exists.
In case of "yes", the actual schedule must also be provided.

Notice that in some cases both answers are valid. In such a case, we do not care
if the procedure outputs "yes" or "no". Suppose we have such a procedure. Then we
use binary search to find the solution. To begin our binary search, we must find an
interval where optimal Cmaxis contained. Notice that (xipj)
/ m is an average load
per machine and maxjpj is the length of the longest jbbl w e can put a bound on
optimum Cmaxas follows:

Lemma 11 Let

then L 5 Cmax< 2L.

Proof: Since the longest job must be completed, we have maxjpj 5 Cmax.Also,
since (xj Pj) /rn is the average load, we have (zj
pj) / m 5 Cmax.Thus, L 5 Cmm.
The upper bound relies on the concept of list scheduling, which dictates that a job
is never processed on some machine, if it can be processed earlier on another machine.
That is, we require that if there is a job waiting, and an idle machine, we must use
this machine to do the job. We claim that for such a schedule Cmax< 2L. Consider
the job that finishes last, say job k. Notice that when it starts, all other machines
are busy. Moreover, the time elapsed up to that point is no more than the average
 load of the machines (see Figure 8). Therefore,

Cj#k Pj
~ m a x I m
+ Plc

Now we have an interval on which to do a (logarithmic) binary search for Cmax.

By TI and T2 we denote lower and upper bound pointers we are going to use in our
binary search. Clearly, T = d m is the midpoint in the logarithmic sense. Based
on Lemma 11, we must search for the solution in the interval [L, ... ,2L]. Since
we use logarithmic scale, we set log TI = log, L, log T2 = log, L +
1 and log T =
+
$(log, TI log, T2).
When do we stop? The idea is to use different value of 6 . That is, the approxima-
tion algorithm proceeds as follows. Every time, the new interval is chosen depending
+
on whether the procedure for the (1 €12)-relaxed decision version returns a "no"
or (in case of "yes") a schedule with makespan 5 (1 E / ~ ) Twhere + , T = d m
and [TI,. .. , T2]is the current interval. The binary search continues until the bounds
TI,T2 satisfy the relation +
5 (1 c), or equivalently 5 2 s.
The number
of iterations required to satisfy this relation is O(lg(l/€)). Notice that this value is a
constant for a fixed E. At termination, the makespan of the schedule corresponding
+
to T2 will be within a factor of (1 E) of the optimal makespan.

In order to complete the analysis of the algorithm, it remains to describe the

+
procedure for the (1 €12)-relaxed decision procedure for any t. Intuitively, if we
look at what can go wrong in list scheduling, we see that it is "governed" by "long"
jobs, since small jobs can be easily accommodated. This is the approach we take,
+
when designing procedure that solves the (1 €12)-relaxed decision version of the
problem. For the rest of our discussion we will denote €12 by t'.
Given {pj), 6' and T, the procedure operates as follows:

Step 1: Remove all (small) jobs with pj 5 E'T.

Step 2: Somehow (to be specified later) solve the (1 + €')-relaxed decision version of the
problem for the remaining (big) jobs.

Step 3: If answer in step 2 is "no", then return that there does not exist a schedule with
makespan 5 T .
+
If answer in step 2 is "yes", then with a deadline of (1 E')Tput back all small
jobs using list scheduling (i.e. greedy strategy), one at a time. If all jobs are
 (l+&)T
deadline deadline

Figure 9: Scheduling "small" jobs.

accommodated then return that schedule, else return that there does not exist a
schedule with makespan 5 T .

Step 3 of the algorithm gives the final answer of the procedure. In case of a "yes" it
is clear that the answer is correct. In case of a "no" that was propagated from Step 2
it is also clear that the answer is correct. Finally, if we fail to put back all the small
jobs we must also show that the algorithm is correct. Let us look at a list schedule
in which some small jobs have been scheduled but others couldn't (see Figure 9).
+
If we cannot accomodate all small jobs with a deadline of (1 $)T, it means
that all machines are busy at time T since the processing time of each small job is
< t'T. Hence, the average load per processor exceeds T. Therefore, the answer "no"
-
is correct.
Now, we describe Step 2 of the algorithm for pj > tlT. Having eliminated the
small jobs, we obtain a constant (when t is fixed) upper bound on the number of
jobs processed on one machine. Also, we would like to have only a small number
of distinct processing times in order to be able to enumerate in polynomial time all
possible schedules. For this purpose, the idea is to use rounding. Let qj be the largest
+
number of the form E'T ~ E ' 5 ~ pjT for some k E N. A refinement of Step 2 is the
following.

2.1 Address the decision problem: Is there a schedule for { q j ) with makespan 5 T?
2.2 If the answer is "no", then return that there does not exist a schedule with
makespan 5 T.
If the answer is "yes", then return that schedule.

The Lemma that follows justifies the correctness of the refined Step 2.

Lemma 12 Step 2 of the algorithm is correct.

Proof: If Step 2.1 returns "no", then it is clear that the final answer of Step 2
should be "no", since qj 5 pj.
If Step 2.1 returns "yes", then the total increase of the makespan due to the
replacement of q j by pj is no greater than ( 1 / c t ) c t 2 ~= ctT. This is true, because
we have at most T/(clT) = 1/c1 jobs per machine, and because pj 5 q j c 1 2 ~by +
definition. Thus, the total length of the schedule with respect to is at most
+ +
T €'T = (1 cl)T.
It remains to show how to solve the decision problem of Step 2.1. We can achieve
this in polynomial time using dynamic programming. Note that the input to this
decision problem is "nice": We have at most P = Ll/~'jjobs per machine, and at
most Q = 1 + IF] distinct processing times. Since c' is considered to be fixed,
we essentially have a constant number of jobs per machine and a constant number
q;, . .. ,qb of processing times. Let n' = i n l , . .. , nQ), where ni denotes the number of
I
jobs whose processing time is qi. We use the fact that the decision problems of PI C,,,
and the bin packing problems are equivalent. Let f (n') denote the minimum number
of machines needed to process n' by time T. Finally, let R = {r' = (rl, . . . ,rQ) :
xZ1 riqi 5 T, ri 5 ni, ri E W. R represents the sets of jobs that can be processed on
a single machine with a deadline of T. The recurrence for the dynamic programming
formulation of the problem is

namely we need one machine to accomodate the jobs in r' E R and f (n' - f )machines
to accomodate the remaining jobs. In order to compute this recurrence we first have
to compute the at most QP vectors in R. The upper bound on the size of R comes
from the fact that we have at most P jobs per machine and each job can have one of
at most Q processing times. Subsequently, for each one of the vectors in R we have
to iterate for nQ times, since ni 5 n and there are Q components in Z. Thus, the
running time of Step 2.1 is 0(n11ft2
(~/c'~)(~/")).
From this point we can derive the overall running time of the pas in a straight-
forward manner. Since Step 2 iterates O(lg(l/t)) times and since t = 2tt, the overall
running time of the algorithm is 0(n1lf (1/ t2)('If) lg ( 1/ t)) .

Randomized Rounding for Multicommodity Flows

In this section, we look at using randomness to approximate a certain kind of mul-
ticommodity flow problem. The problem is as follows: given a directed graph G =
(V,E), with sources si E V and sinks ti E V for i = 1 , . . . , k, we want to find a
path Pi from si to ti for 1 5 i 5 k such that the "width" or "congestion" of any
edge is as small as possible. The "width" of an edge is defined to be the number of
paths using that edge. This multicommodity flow problem is NP-complete in general.
The randomized approximation algorithm that we discuss in these notes is due to
Raghavan and Thompson [24].

Reformulating the problem

The multicommodity flow problem can be formulated as the following integer pro-
gram:

Min W
subject to:

C xi(V, W) - C xi(w, V) =
w w 0 otherwise

(v, w) E E.

Notice that constraint (6) forces the xi to define a path (perhaps not simple) from si
to ti. Constraint (7) ensures that every edge has width no greater than W, and the
overall integer program minimizes W .
We can consider the LP relaxation of this integer program by replacing the con-
straints xi(v, w) E {0,1) with xi(v, w) 2 0. The resulting linear program can be
solved in polynomial time by using interior point methods discussed earlier in the
course. The resulting solution may not be integral. For example, consider the multi-
commodity flow problem with one source and sink, and suppose that there are exactly
i edge-disjoint paths between the source and sink. If we weight the edges of each path
by (i.e. set x(v, w) = for each edge of each path), then WLp= +. The value WLp
can be no smaller: since there are i edge-disjoint paths, there is a cut in the graph
with i edges. The average flow on these edges will be f , so that the width will be at
least +.
The fractional solution can be decomposed into paths using flow decomposition, a
standard technique from network flow theory. Let x be such that x 2 0 and
a if v = s;
C X ( V , W) - C X(W, V) =
w W 0 otherwise.

Then we can find paths P I , . . . ,Pi from si to ti such that

To see why we can do this, suppose we only have one source and one sink, s and t .
Look at the "residual graph" of x: that is, all edges (v, w) such that x(v, w) > 0.
Find some path PI from s to t in this graph. Let a1 = min(,,,)Ep, x ( v , w ) . Set

(v, W) E Pl
otherwise.

We can now solve the problem recursively with a' = a - al.

7.2 The algorithm

We now present Raghavan and Thompson's randomized algorithm for this problem.

1. Solve the LP relaxation, yielding WLP.

2. Decompose the fractional solution into paths, yielding paths Pij for i = 1,. . . , k
and j = 1 , . . . ,ji (where Pij is a path from si to t i ) ,and yielding aij > 0 such
that aij = 1 and

3. (Randomization step) For all i, cast a ji-faced die with face probabilities aij . If
the outcome is face f , select path Pij as the path Pi from si to ti.

We will show, using a Chernoff bound, that with high probability we will get small
congestion. Later we will show how to derandomize this algorithm. To carry out the
derandomization it will be important to have a strong handle on the Chernoff bound
and its derivation.

C hernoff bound
For completeness, we include the derivation of a Chernoff bound, although it already
appears in the randomized algorithms chapter.

Lemma 13 Let Xi be independent Bernoulli random variables with probability of

success pi. Then, for all a > 0 and all t > 0, we have

Proof:
for any u > 0. Moreover, this can be written as P r [ Y > a] with Y 2 0. From
Markov's inequality we have

for any nonnegative random variable. Thus,

= >~t ]
P r [ ~ f Xi <
- e-"tE [eaxi
- e-*t nf=, E [emxi] because of independence.

The equality then follows from the definition of expectation.

+
Setting t = (1 P ) E [ C ,Xi] for some P > 0 and u = ln(1 + P ) , we obtain:
Corollary 14 Let Xi be independent Bernoulli random variables with probability of
k
success pi, and let M = E [ c ~ =XI]
~ = Ci=, pi. Then, for all ,6 > 0, we have

< (1+~)-('+"l"n~
I I".
k
eP
> (1 + P ) M [(1
i=l i=l < [ ( 1+ P ) ( l + P )
The second inequality of the corollary follows from the fact that

7.4 Analysis of the R-T algorithm

Raghavan and Thompson show the following theorem.

Theorem 15 Given e > 0, if the optimal solution to the multicommodity flow prob-
lem W* has value W* = fl(1og n) where n = IVI, then the algorithm produces a
+
solution of width W 5 W* c J m with probability 1 - e (where c and the con-
stant in fl(1ogn) depends on e, see the proof).

Proof: Fix an edge (v, w ) E E. Edge (v, w ) is used by commodity i with proba-
bility pi = Cj:(u,w)~pi, aij. Let Xi be a Bernoulli random variable denoting whether
or not (v, w ) is in path Pi.Then W(v, w ) = Xi, where W(v, w ) is the width of
edge (v, w ). Hence,

Now using the Chernoff bound derived earlier,

Assume that p 5 1. Then, one can show that

Therefore, for

we have that
P.[W(v, w) 2 (1 + P)W*] I
. ,n
&

Notice that our assumption that 5 1 is met if

W* 2 61nn -31n~.

For this choice of ,B, we derive that

(1 + p)W* = W* + +W*ln-. n2 &

We consider now the maximum congestion. We have

proving the result.

Derandomization
We will use the method of conditional probabilities. We will need to supplement this
technique, however, with an additional trick to carry through the derandomization.
This result is due to Raghavan [23].
We can represent the probability space using a decision tree. At the root of the
tree we haven't made any decisions. As we descend the tree from the root we represent
the choices first for commodity 1, then for commodity 2, etc. Hence the root has jl
children representing the jl possible paths for commodity 1. Each of these nodes has
j2 children, one for each of the j2 possible paths for commodity 2. We continue in
the manner, until we have reached level k . Clearly the leaves of this tree represent
all the possible choices of paths for the k commodities.
A node at level i (the root is at level 0) is labeled by the i choices of paths for
commodities 1 . .. i : Il . . . I;. Now we define:

I
Il for commodity 1 -
12 for commodity 2
g(l1.l)=Pr max W ( v , w ) > ( l + , B ) W * .
(v,w)€E
1; for commodity i -
By conditioning on the choice of the path for commodity i, we obtain that

If we could compute g(ll, 12, .. . ) efficiently, we could start from g(0) and by select-
ing the minimum at each stage construct a sequence g(0) >
g(ll) >
g(ll, 12) >
>
. . . g(ll, 12, . . . , lk). Unfortunately we don't know how to calculate these quantities.
Therefore we need to use an additional trick.
Instead of using the exact value g, we shall use a pessimistic estimator for the
probability of failure. From the derivation of the Chernoff bound and the analysis of
the algorithm, we know that

where the superscript on Xi denotes the dependence on the edge (v,w), i.e. =1 x!"'~)
if (v,w) belongs to the path Pi. Letting h(ll, . . . , 1;) be the RHS of (10) when we
condition on selecting path Pjrl for commodity j, j = 1, . . . ,i, we observe that:

1. h(ll, . .. , li) can be easily computed,

2. g(ll, . . . , 1;) 5 h(ll, . . . , li) and

Therefore, selecting the minimum in the last inequality at each stage, we construct
a sequence such that 1 > E > h(0) > >
h(ll) >_ h(ll, 12) >_ . . . >
h(ll, 12,. . . , lk)
g(h, 12,. . . , lk). Since g(ll, 12, .. . , lk)is either 0 or 1 (there is no randomness involved),
we must have that the choice of paths of this deterministic algorithm gives a maximum
congestion less than (1 P)W*. +
Mult icommodity Flow
Consider an undirected graph G = (V, E) with a capacity u, on each edge. Suppose
that we are given k commodities and a demand for fi units of commodity i between
two points si and ti. In the area of multicommodity flow, one is interested in knowing
whether all commodities can be shipped simultaneously. That is, can we find flows
of value fi between si and ti such that the sum over all commodities of the flow on
each edge (in either direction) is at most the capacity of the edge.
There are several variations of the problem. Here, we consider the concurrent flow
problem: Find a* where a* is the maximum a such that for each commodity we can
ship afiunits from si to ti. This problem can be solved by linear programming since
all the constraints are linear. Indeed, one can have a flow variable for each edge and
each commodity (in addition to the variable a), and the constraints consist of the
flow conservation constraints for each commodity as well as a capacity constraint for
every edge. An example is shown in figure 8. The demand for each commodity is 1
unit and the capacity on each edge is 1 unit. It can be shown that a* = 2.

Figure 10: An example of the multi-commodity flow problem.

When there is only one commodity, we know that the maximum flow value is equal
to the minimum cut value. Let us investigate whether there is such an analogue for
multicommodity flow. Consider a cut (S,S). As usual S(S) is the set of edges with
exactly one endpoint in S . Let,

Since all flow between S and S must pass along one of the edges in S(S) we must
have,

where u(S(S)) = xeE6(S)

u(S S
tie. T he multicommodity cut problem is to find a set S
which minimizes #. We let B* be the minimum value attainable and so we have
a* 5 p*. But, in general, we don't have equality. For example, in Figure 8, we have
p* = 1. In fact, it can be shown that the multicommodity cut problem is NP-hard.
We shall consider the following two related questions.
1. In the worst case, how large can 5be?
2. Can we obtain an approximation algorithm for the multicommodity cut prob-
lem?
In special cases, answers have been given to these questions by Leighton and Rao [19]
and in subsequent work by many other authors. In this section, we describe a very
recent, elegant and general answer due to Linial, London and Rabinovitch [20]. The
technique they used is the embedding of metrics. The application to multicommodity
flows was also independently obtained by Aumann and Rabani [3].
We first describe some background material on metrics and their embeddings.
Definition 8 (X,d) is a metric space or d is a metric on a set X if

2. 'v'x, y : d(x, y) = d(y,x).

3. 'v'x,y , z : d(x, y) + d(y, z) 2 d(x, z).
Strictly speaking we have defined a semi-metric since we do not have the condition
d(x, y ) = 0 +-x = y. We will be dealing mostly with finite metric spaces, where X is
finite. In Rn then the following are all metrics:

d(x,y) = Ilx-yll2 = d ~ ( x i - y i ) ~e2metric

d(x,y) = llx-ylll = CIxi-yiI el metric
= ) llx-yllm
d ( ~ , ~ = maxi1xi -yil t, metric
~ ( x , Y )= IIx - YIIP = ( x i - yi 1 ) 6 metric
Definition 9 (X,d) can be embedded into (Y, t) if there exists a mapping (o : X + Y
which satisfies 'v'x,y : l((o(x),(o(y)) = d(x, y) .
Definition 10 (X,d) can be embedded into (Y,t) with distortion c if there exists a
mapping cp : X -+Y which satisfies Vx, y : d(x, y ) 5 t((o(x),(o(y)) 5 cd(x, y).
The following are very natural and central questions regarding the embedding of
metrics.
ISIT-4: Given a finite metric space (X,d), can it be embedded into (Wn, 4 )for
some n?
EMBED-lp: Given a finite metric space (X,d) and c 2 1, find an embedding of
(X, d) into (Wn,tp) with distortion at most c for some n.
As we will see in the following theorems, the complexity of these questions depend
critically on the metric themselves.
Theorem 16 Any (X,d) can be embedded into (Rn,t,) where n = 1x1. (Thus, the
answer to ISIT-t, is always "yes".)
Proof: We define a coordinate for each point z E X. Let d(x,z) be the z
coordinate of x. Then,

because of the triangle inequality.

Theorem 17 ISIT-l2 E P. (i.e., ISIT-l2 can be answered in polynomial time.)

Proof: Assume that there exists an embedding of {1,2, .. . , n } into {vl = 0, v2, . .. ,v,}.
Consider one such embedding. Then,

d2 ( i ) = v -v
2
= (vi - vj)(vi - v j ) = vf - 2vi - vj + vj.
2

But v: = d2(1,i) and v: = d2(1,j) which means that,

We now construct M = (mij) where,

Hence if M is not positive semi-definite then there is no embedding into 12. If M is

positive semidefinite then we carry out a Cholesky decomposition on M to express
M as M = V V ~ From. the rows of V we can obtain an embedding into (Rn,12).

Theorem 18 ISIT-ll is NP-complete.

This theorem is given without proof. The reduction is from MAX CUT, since as we
will see later there is a very close relationship between 11-embeddable metrics and
cuts. We also omit the proof of the following theorem.

Theorem 19 Let X & Rn. (X,12)can be embedded into (Rm,e l) for some m.

The converse of this theorem is not true as can be seen from the metric space
( N O ? o), (-1, O)? (17 01, (07 l ) } ? l l ) .

Reducing multicommodity flow/cut questions to embed-

ding questions
In this section, we relate a* and ,4?* through the use of metrics.

Claim 20

min &,~)EE ~ x y t ( xY,

ll-embeddable metrics (V, l) fil(~it i) '

Proof:
(2)Given S, let

Let l be the ll metric on the line, i.e. l ( a , 6) = la - bl. Then,

u(b(S)) = C uXY% Y)
(x,Y)EE
k
f (s) = C
i=
fil(si
1
ti)

since l ( x , y) = 1 if and only if z is separated from y by S and 0 otherwise.

( 5 )We can view any ll embeddable metric l as a combination of cuts. See figure 11
for the 2-dimensional case.

Figure 11: Viewing an &-metric as a combination of cuts.

For any set S define a metric is by,

1 if x, y are separated by S
Is = { 0 otherwise.

Then we can write l as,

where the ai's are nonnegative. Hence,

C ( x , y ) E ~uxyl(x, Y)
-
Ci a i ~ ( b ( S i ) 2
) min u(b(S))
~ j " =fiP(si
1 ti) aif (si) f(S) '
Claim 21

a* = min
&,~)EE ~xYd(x7y )
l , -embeddable metrics (V,d ) Ci fid(si ti)
Note that by theorem 16 we actually minimize over all metrics.
Proof:

( 5 )For any metric d let the volume of an edge ( x ,y ) be u X y d ( xy).

, The total volume
of the graph is C(x,y)EE , If we send a fraction a of the demand then the
u X y d ( xy).
amount of volume that we use is at least a X i fid(si, ti).Hence a X i fi d(si,ti) 5
C ( x , y )~EX~y d ( xY,) -
(2)We use the strong duality of linear programming. a* can be formulated as a
linear program in several different ways. Here we use a formulation which works
well for the purpose of this proof although it is quite impractical. Enumerate
the paths from si to ti, let Pij be the j t h such path and let xij be the flow on
Pij. The linear program corresponding to multicommodity flow is,

Max a
subject to:

The dual of this linear program is:

Min x
eEE
uele

subject to:

The second constraint in the dual implies that hi is at most the shortest path
length between si and ti with respect to l,. By strong duality if l is an optimum
 solution to the dual then,

j)
C(i,j)EEuijd(i,
2
Cf=1fid(si ti) '
where d(a, b) represents the shortest path length with respect to l,. The first
inequality holds because fihi is constrained to be at least 1.

Linial, London, and Rabinovitch and Aumann and Rabani use the following strat-
egy to bound 5
and approximate the minimum multicommodity cut.

1. Using linear programming, find a* and the corresponding metric d as given in

claim 2 1.

2. Embed d into (Rm,e l) with distortion c. Let l be the resulting metric.

By claim 20 this shows that 55 c since,

C ( x , y ) E ~~ Xyl(x,Y) < c C ( ~ , ~ ) E~Xyd(x,
E Y)
P* 5 - = CQ*.
Cf=1fie(si ti) Cf=1fid(si, ti)
In order to approximate the minimum multicommodity cut, we can use the proof of
claim 20 to decompose !into cuts. If S is the best cut among them then,

u ( ~ ( S<
) )C ( x , yUXY!(X~
) ~ ~ Y)
-
f (S) Cf=1fie(si ti) '

Our remaining two questions are:

a How do we get an embedding of d into ?! Equivalently, how can we embed !

,
into ll.

a What is c?

Embedding metrics into el

The following theorem is due to Bourgain.

Theorem 22 For all metrics d on n points, there exists an embedding of d into el

which satisfies:
4x7 Y) I IIx - Y Ill I O(1og n)d(x, y).

Approx-44
Proof: Let k range over { 1 , 2 , 4 , 8 , . . . , 2 j , . . . ,2P} where p = llog n ] . Hence we
have p + 1 = O(1og n ) different values for k. Now choose n k sets of size k. At first
take all sets of size k , i.e., n k = (L). Introduce a coordinate for every such set. This
implies that points are mapped into a space of dimension C:=, nzl < 2n. For a set A
of size k the corresponding coordinate of a point x is,

where d ( x , A) = minzEAd ( x ,z ) and a is a constant which we shall determine later.

Suppose that d ( x , A ) = d ( x , s ) and d(y ,A) = d ( y,t ) , where s and t are in A. Then,

Exchanging the roles of x and y, we deduce that ld(x,A) - d ( y,A) IId ( x , y ). Hence,

We now want to prove that 1 lx - y 1 ll 2 d ( x ,y). Fix two points x and y and define,

B ( x , r ) = { z :d(x,z) 5 r } ,
B(X,T) = { z :d(x,z)< r),
po = 0,
pt = infir : IB(x,r)l > 2t, l B ( y , r ) l > 2t).
Let l be the least index such that pl 2 y.Redefine p' so that it is equal to y.
I
Observe that for all t either B ( x , p t ) 1 < 2t or IB(y, p t ) 1 < 2t. Since B ( x , pl-1) n
+-
B( y , pl-1) = PI we have 2'-I +2'-l 5 n l 5 p. Now fix k = 2j where p- 1 2 j 2 p - l
and let t = p - j (thus, 1 I t 5 I ) . By our observation we can assume without loss
I I
of generality that B ( X , pt) < 2t. Let A be a set of size k and consider the following
two conditions.

If 1. and 2. hold then d ( x , A ) 2 pt and d ( y , A ) 5 pt-, and so l d ( x , A ) - d ( y , A ) I 2

pt - pt-1. Let

Rk= { A : IAl = k and A satisfies conditions 1. and 2.)

Lemma 23 For some constant P > 1 (independent of k ) , there are at least 7 sets
of size k which satisfy conditions 1. and 2) i.e. lRkl 2 7.
From this lemma we derive,

Hence if we choose a = 4P then we have 1 lx - y 1 1 2 d ( x , y ) . We now have to prove

lemma 23.
Proof of lemma 23: Since I B ( x , pt)l < 2t, IB(y, pt-l)l 2 2t-1 and we are
considering all sets of size k the following is a restatement of the lemma: Given
disjoint sets P and Q with a = lPl < ztand b = IQI 2 2t-1, if E is the event that
a uniformly selected A misses P and intersects Q then Pr[E] 2 $. We calculate this
probability as follows:

As an approximation (this can be made formal), we replace ( 1 - *)

b y e-"in, and
( 1 - *) by e-(a+b)/n.Thus,
n-3
This for example shows that if a , b and k are all fithen this probability is a constant,
which may seem a bit paradoxical. Using our bounds on a and b, we get

We now choose 3 z (:-'( 1 - em$))-' and the proof is complete.

Bourgain's proof is not quite algorithmic since the dimension is exponential.
Linial, London and Rabinovitch just sample uniformly with n k = O(1og n ) and show
that with high probability the embedding has the required properties. This follows
from a Chernoff bound.
We have thus shown that the distortion c can be chosen to be O(1og n ) . We can
do even better by proving the following variant to Bourguain's theorem.

Theorem 24 Let d be a metric on a set V of n points. Suppose that T & V and

IT1 = k . Then there exists an embedding of d into el which satisfies:

In order to prove this theorem we restrict the metric to T and then embed the
restricted metric. If we look at the entire vertex set V then the first part of the
original proof still works.
This new theorem is enough to show that 55 O(1og k) and to approximate the
multicommodity cut to within O(1og k ) . This result is best possible in the sense that
we can have 5 = @(logk ) .

References
[ I ] S. Arora, C. Lund, R. Motwani, M. Sudan, and M. Szegedy. Proof verification
and hardness of approximation problems. In Proceedings of the 33rd Annual
Symposium on Foundations of Computer Science, pages 14-23, 1992.

[2] S. Arora and S. Safra. Probabilistic checking of proofs. In Proceedings of the

33rd Annual Symposium on Foundations of Computer Science, pages 2-13, 1992.

[3] Y. Aumann and Y. Rabani. An O(1og k ) approximate min-cut max-flow theorem

and approximation algorithm. Manuscript, 1994.

[4] R. Bar-Yehuda and S. Even. A linear time approximation algorithm for the
weighted vertex cover problem. Journal of Algorithms, 2:198-203, 1981.
 [5] M. Bellare and M. Sudan. Improved non-approximability results. In Proceedings
of the 26th Annual ACM Symposium on Theory of Computing, pages 184-193,
1994.

[6] N. Christofides. Worst-case analysis of a new heuristic for the travelling salesman
problem. Technical Report 388, Graduate School of Industrial Administration,
Carnegie Mellon University, Pittsburgh, PA, 1976.

[7] W. F. de la Vega and G. S. Luecker. Bin packing can be solved within (1 + t) in

linear time. Combinatorica, 1(4), 1981.

[8] J. Edmonds. Maximum matching and a polyhedron with 0,l-vertices. Journal

of Research of the National Bureau of Standards B, 69B:125-130, 1965.

[9] R. Fagin. Generalized first-order spectra, and polynomial-time recognizable sets.

In R. Karp, editor, Complexity of Computations. AMS, 1974.

[lo] H. N. Gabow. Data structures for weighted matching and nearest common ances-
tors with linking. In Proceedings of the 1st ACM-SIAM Symposium on Discrete
Algorithms, pages 434-443, 1990.

[ll] H. N. Gabow, M. X. Goemans, and D. P. Williamson. An efficient approxima-

tion algorithm for the survivable network design problem. In Proceedings of the
Third MPS Conference on Integer Programming and Combinatorial Optimiza-
tion, pages 57-74, 1993.

[12] H. N. Gabow and R. E. Tarjan. Faster scaling algorithms for general graph
matching problems. Technical Report CU-CS-432-89, University of Colorado,
Boulder, 1989.

[13] M. X. Goemans and D. P. Williamson. A general approximation technique for

constrained forest problems. In Proceedings of the 3rd Annual ACM-SIAM Sym-
posium on Discrete Algorithms, pages 307-31 6, 1992.

[14] M. X. Goemans and D. P. Williamson. Improved approximation algorithms

for maximum cut and satisfiability problems using semidefinite programming.
In Proceedings of the 26th Annual ACM Symposium on Theory of Computing,
pages 422-431, 1994.

[15] A. Haken and M. Luby. Steepest descent can take exponential time for symmetric
connection networks. Complex Systems, 2: 191-196, 1988.

[16] D. Hochbaum. Approximation algorithms for set covering and vertex cover prob-
lems. SIAM Journal on Computing, 11:555-556, 1982.
[17] D. Hochbaum and D. Shmoys. Using dual approximation algorithms for schedul-
ing problems: theoretical and practical results. Journal of the ACM, 34(1), Jan.
1987.

[18] N. Karmarkar and R. Karp. An efficient approximation scheme for the one-
dimensional bin-packing problem. In Proceedings of the 23rd Annual Symposium
on Foundations of Computer Science, 1982.

[I91 T. Leighton and S. Rao. An approximate max-flow min-cut theorem for uniform
mult icommodity flow problems with applications to approximat ion algorithms.
In Proceedings of the 29th Annual Symposium on Foundations of Computer Sci-
ence, pages 422-431, 1988.

[20] N. Linial, E. London, and Y. Rabinovich. The geometry of graphs and some of
its algorithmic applications. In Proceedings of the 35th Annual Symposium on
Foundations of Computer Science, 1994.

[21] C. H. Papadimitriou and M. Yannakakis. Optimization, approximation, and

complexity classes. Journal of Computer and System Sciences, 43:425-440, 1991.

[22] S. Poljak. Integer linear programs and local search for max-cut. Preprint, 1993.

[23] P. Raghavan. Probabilistic construction of deterministic algorithms: approxi-

mating packing integer programs. Journal of Computer and System Sciences,
37:130-143, 1988.

[24] P. Raghavan and C. D. Thompson. Randomized rounding: a technique for

provably good algorithms and algorithmic proofs. Combinatorica, 7:365 374.
-

1987.