Lecture Notes in

Computational Optimization
Jianer Chen
Department of Computer Science
Texas A&M University
College Station, TX 77843-3112
U. S. A.
[email protected]
December, 1995
 Preface
This manuscript consists of lecture notes for CPSC-669 Computational
Optimization as it was taught by me in the Fall of 1995 at Department of
Computer Science, Texas A&M University. The notes were rst taken by
students in the class then were rewritten by myself. The notes were not
meant at all to be in polished form and they probably contain many errors.
I will appreciate that readers let me know their corrections and comments.
Because of the time limit, I was not able to cover many other recent
interesting and important results in this set. The rst few of them in my
mind are the probabilistic method and derandomization, recent improved
approximation algorithms for Max-Sat and Max-Cut, and approximability
of Steiner trees. The discussion on linear programming should certainly be
in more detail and in more depth. My plan is to add at least these topics in
the next revision.
Help from the following list of scribes is acknowledged: M. Chatterjee,
X. Chen, S. Lu, L. Shao, B. Varanasi, J. Walter, W. Zhang, and H. Zheng.
I also appreciate encouraging discussion and comments from Professors D.
Friesen and C. Papadimitriou.
 CPSC-669 Computational Optimization
Lecture #1, August 30, 1995
Lecturer: Professor Jianer Chen
Scribe: Jennifer Walter
Revision: Jianer Chen
1 Introduction
Most computational optimization problems come from practice in industry
and other elds.
Denition 1.1 An optimization problem Q is a 4-tuple hIQ, SQ, fQ, optQi,
where IQ is the set of input instances, SQ is a function such that for each
input x 2 IQ , SQ (x) is a set of solutions to x, fQ is the objective function
such that for each pair x 2 IQ and y 2 SQ (x), fQ (x; y ) is an integer, and
optQ 2 fmax; ming species the problem to be a maximum problem or a
minimum problem.
Therefore, an optimization problem can be dened as follows: given an
input instance x, nd a solution y in SQ (x) such that the objective function
value fQ (x; y ) is optimized (maximized or minimized depending on optQ )
among all solutions in SQ(x).
Remark 1.2 The 4-tuple must satisfy the following conditions for an opti-
mization problem:
1. It should be testable in polynomial time whether a given x is an input
instance of Q.
2. It should be testable in polynomial time given x and y whether y is a
solution in S (x).
3. The objective function f (x; y ) should be computable in polynomial
time, given x 2 IQ and y 2 SQ (x).
Examples of combinatorial optimization problems:
1. Minimum Spanning Tree
2. Shortest Path
3. Knapsack

1
 4. Bin Packing
5. Vertex Cover
6. Traveling Salesman Problem
This list is not exhaustive. There are many other optimization problems.
Example 1.3 How do we formulate the Minimum Spanning Tree problem
using the above formulation?
By using the denition of an optimization problem, we can formulate the
MST problem as follows:
IQ : weighted graph G
SQ: all spanning trees of G
fQ: fQ (G; T ) = sum of weights of edges of T , a spanning tree of G
optQ : min
Example 1.4 How do we formulate the Shortest Path problem?
IQ : weighted graph G with two specied vertices u; v
SQ: all paths connecting vertices u; v in G
fQ: fQ (G; u; v; p) = length of p, a path connecting u; v
optQ : min
Example 1.5 How do we formulate the Knapsack problem?
IQ : Set S = fx1 ; x2; : : :; xng, where each x has size si and prot fi . Bound
B on sizePis also dened.
SQ: S S , xiP 2Ssi  B
fQ: fQ (S; S) = xi 2Sfi
optQ : max
Example 1.6 How do we formulate the Bin Packing problem?
IQ : Set S = fx1; x2; : : :; xng, where 0 < xi < 1.
SQ: Partition P of S into S1 [ S2 [ : : : [ Sr such that Px2Si x  1
fQ: fQ (S; P ) = r
optQ : min
Example 1.7 How do we formulate the Vertex Cover problem?

2
IQ : A graph G = (V; E ).
SQ: A subset S of V such that every edge e in E has at least one end
in S .
fQ : fQ (G; S ) = the number of vertices in S .
optQ : min
Example 1.8 How do we formulate the Traveling Salesman problem?
IQ : A weighted complete graph G = (V; E ).
SQ: A path P in G that goes through all vertices of G.
fQ : fQ (G; P ) = the weight of the path P .
optQ : min
Examples 1.3 and 1.4 can be solved in polynomial time. Examples 1.5
to 1.8 are known to be NP-hard, which means it is unlikely to have ecient
algorithms for solving them precisely. For these problems, we will discuss
ecient approximation algorithms that nd solutions \close" to the optimal
ones. We will see that for Knapsack problem, there is a very good approx-
imation algorithm that produce solutions arbitrarily close to the optimal
solutions. For Bin Packing problem and Vertex Cover, we will see that ap-
proximation algorithms of constant ratio will be possible while it is unlikely
for them to have further better approximation algorithm. For Traveling
Salesman problem, we will see that any reasonable approximation will be
infeasible.
The course will start with optimization problems that can be solved
in polynomial time. Examples are Maximum Flow, Matching, and Linear
Programming. Then we discuss approximation algorithms on NP-hard op-
timization problems. We rst discuss techniques that approximate NP-hard
optimization problems with solutions that are arbitrarily close to optimal
solutions. This class of optimization problems includes Knapsack and many
scheduling problems. Then we present approximation algorithms with con-
stant ratio for certain optimization problems and show that no much better
approximation algorithms are possible for these problems. Bin Packing and
Vertex Cover belong to this class. We will also discuss optimization problems
such as Traveling Salesman Problem, which are very hard to approximate.

3
 CPSC-669 Computational Optimization
Lecture #2, September 1, 1995
Lecturer: Professor Jianer Chen
Scribe: Jennifer Walter
Revision: Jianer Chen
2 Max-Flow Problem
Denition 2.1 A ow graph G = (V; E ) is a directed and positively weighted
graph with two distinguished vertices s (the source) and t (the sink). The
weight on an edge (u; v ) is called the capacity of the edge, and is designated
by cap(u; v ). If there is no edge from vertex u to vertex v , then we dene
cap(u; v) = 0.
Remark 2.2 Edges can be directed into the source and out of the sink.
Denition 2.3 A ow f on a ow graph G = (V; E ) is a function on pairs
of vertices of G satisfying the following conditions:
1. For all u; v 2 V , cap(u; v )  f (u; v ).
2. For all u; v 2 V , f (u; v ) = f (v; u).
P
3. For all u 6= s; t, v2V f (u; v ) = 0.
Question 2.4 What is the ow value from u to v if there is no edge between
u and v ?
By the denition, if there is no edge between u and v , then we have cap(u; v ) =
cap(v; u) = 0. By the rst condition of a ow f , we must have cap(u; v ) 
f (u; v) and cap(v; u)  f (v; u). These together with the second condition of
the ow f (u; v ) = f (v; u) give immediately f (u; v ) = 0.
Remark 2.5 Note the following about capacities and ows:
 cap(u; v) is always dened.
 If cap(u; v) = 0, then f (u; v) can be negative.
 cap(u; v) is in general not equal to cap(v; u).
4
Denition 2.6 Given a ow graph G = (V; E ) and given a ow f on G, the
residual graph Gf = (V; E 0) of G (with respect to the ow f ) has the same
vertex set as G. Moreover, for each vertex pair u; v , if cap(u; v ) > f (u; v ),
then (u; v ) is an edge in Gf with capacity cap(u; v ) f (u; v ).
Remark 2.7 New edges may be created in the residual graph Gf that were
not originally present in the original graph G.
Remark 2.8 Max-Flow problem can be formulated using our denition of
optimization problems as a 4-tuple Max-Flow = hI; S; f; opti
I : ow graphs G with source s and sink t
S : S (G) is thePset of valid ows f on G
f : f (G; f ) = v2V f (s; v )
opt: max
Remark 2.9 The goal in the Maximum Flow Problem is to nd the max-
imum ow from source to sink. Solving the Max-Flow problem involves
nding paths from s to t and pushing the maximumP ow over those paths.
Formally, the goal of Max-Flow is to maximize v2V f (s; v ), the amount of
ow coming P out of the source. Alternatively, the goal could be specied as
maximizing w2V f (w; t), the amount of ow going into the sink. It can be
proved that these two denitions are equivalent. The proof is not very hard
and left to the students.

5
 CPSC-669 Computational Optimization
Lecture #3, September 4, 1995
Lecturer: Professor Jianer Chen
Scribe: Mitrajit Chatterjee
Revision: Jianer Chen
3 Max-Flow Problem (Contd.)
Theorem 3.1 Let G be a ow-graph and let f be a ow in G. The ow f
is a maximum ow in G if and only if the residual graph Gf has no positive
ow.
proof.
()). Assume
P that there is a positive ow f  in the residual graph Gf ,
i.e. jf j = v2V f  (s; v ) > 0.

Dene a function f + on each pair (u; v ) of vertices in the ow-graph G
as follows:
f + (u; v ) = f (u; v ) + f (u; v)
Claim: f + is a valid ow in G.
Proof for the Claim: A ow is valid if it satises all the three conditions as
described in Denition 2.3. The conditions are veried as follows.
(a) For all u; v 2 V , cap(u; v )  f + (u; v ):
We compute the value cap(u; v ) f + (u; v ). By denition we have
cap(u; v) f + (u; v ) = cap(u; v ) f (u; v ) f (u; v )
Now by the denition of capf , we have cap(u; v ) f (u; v ) = capf (u; v ).
Moreover, since f (u; v ) is a valid ow in the residual graph Gf , capf (u; v )
f (u; v )  0. Consequently, we have cap(u; v ) f +(u; v)  0. The condition
is thus satised.
(b) For all u; v 2 V , f + (u; v ) = f + (v; u):
Since both f (u; v ) and f  (u; v ) are valid ows in ow-graphs G and Gf ,
respectively, we have f (u; v ) = f (v; u) and f  (u; v ) = f  (v; u). Thus,
f + (u; v ) = f (u; v ) + f (u; v ) = f (v; u) f (v; u) = f + (v; u)
P
(c) For all u 6= s; t, v2V f + (u; v ) = 0:
6
 Again, since both f (u; v ) and f  (u; v ) are valid ows in ow-graphs G
and Gf , respectively, we have for all u 6= s; t
X X
f (u; v) = f (u; v ) = 0
v2V v2V
Thus X X X
f + (u; v) = f (u; v ) + f  (u; v) = 0
v2V v2V v2V
f+
Thus, the function satises all three conditions for a ow in G and
is a valid ow in the ow-graph G. Now we compute jf + j and note that
jf j > 0, we get
X X X
jf + j = f + (s; v ) = f (s; v ) + f  (s; v) = jf j + jf  j > jf j
v2V v2V v2V
Hence f is not a maximum ow in G.
((). Here, we assume that f is not a maximum ow in G. Let fmax be
a maximum ow in G. Thus, jfmaxj jf j > 0. Now dene a function f on
each pair (u; v ) of vertices in the ow-graph Gf as follows.
f (u; v ) = fmax(u; v ) f (u; v)
Claim: f is a valid ow in Gf .
Proof for the Claim: Again we verify the three conditions of a ow in Gf .
(a) For all u; v 2 V , capf (u; v )  f (u; v ):
capf (u; v) f (u; v ) = cap(u; v) f (u; v) f (u; v )
Note that f (u; v ) + f (u; v ) = fmax(u; v ). Since fmax is a valid ow in G,
we have cap(u; v ) fmax(u; v )  0. Consequently, we have capf (u; v )
f (u; v )  0.
(b) For all u; v 2 V , f (u; v ) = f (v; u):
f (u; v ) = fmax(u; v ) f (u; v ) = fmax (v; u) + f (v; u) = f (v; u)
P
(c) For all u 6= s; t, v2V f (u; v ) = 0:
X X X
f (u; v ) = fmax(u; v ) f (u; v ) = 0
v2V v2V v2V

7
 This, f is a valid ow in the ow-graph Gf . Moreover, since we have
X X X
jf j = f (s; v ) = fmax (s; v ) f (s; v ) = jfmaxj jf j > 0
v2V v2V v2V
We conclude that the residual graph Gf has a positive ow.
This completes the proof of the theorem.
Theorem 3.1 ensures the correctness of the following algorithm.
Algorithm 3.1 Max-Flow
Input: A flow-graph . G
Output: A maximum flow on . f G
1. Let ( f u; v
) = 0 for all pairs ( ) of vertices inu; v G;
2. Construct the residual graph f ; G
3. whilethere is a positive flow  in f f G do
Construct a positive flow  in f ; f G
Let = +  be the new flow on .
f f f G
Construct the residual graph f; G
Remark 3.1 Whenever there is a positive ow f  in Gf , there is at least
one directed path in Gf from s to t on which all the edges have a positive
capacity. There can be several approaches to nd such paths in the residual
graph Gf . An algorithm by Ford-Fulkerson nds a path of maximum capac-
ity. This algorithm is ecient in most cases, but can perform badly in some
cases. In this context, Dinic's (Dinitz) algorithm has a stronger bound on
the time complexity. This algorithm tries to nd the shortest path from s to
t. The path length is based on the number of edges in the path. The short-
est path can be determined by using breadth rst search (BFS) algorithm.
In each iteration of the while loop in Algorithm 3.1, Dinic's algorithm will
push the ow through all the shortest paths, so that in the next iteration,
the length of the shortest path increases at least by one. Dinic's algorithm
and its analysis will be presented in the next lecture.

8
 CPSC-669 Computational Optimization
Lecture #4, September 6, 1995
Lecturer: Professor Jianer Chen
Scribe: Mitrajit Chatterjee
Revision: Jianer Chen
4 Max-Flow Problem (Contd.)
Algorithm 4.1 BFS Dinic
Input: A flow-graph . G
Output: A leveled graph L V ;E
= ( L L) containing all shor-
test paths in from G to . s t
1. C level = 1; f
C level = current level g
v
2. For all vertices , level[ ] = + 1; v n
s
3. level[ ] = 0; Q ; L= s V ; fsg
fQis a queue. g
4. while Q <
is non-empty and C level level[ ] t do
v Q ;
if(C level < v then
level[ ]) C level = level[ ]; v
for v; w
each edge ( G do
) in
if w n
(level[ ] = + 1)
then V V Sfwg Q w
L= L ; ;
if w > v then
E E Sf v; w g
(level[ ] level[ ])
L= L ( );
w
level[ ] = level[ ] + 1; v
Remark 4.1 The above algorithm is a modication of the famous breadth
rst search algorithm. The analysis can be performed similarly as for
breadth rst search. Thus, we conclude that the time complexity of the
algorithm is O(e), where e is the number of edges in the ow-graph G. This
algorithm stops either when it reaches t (in this case, the leveled graph L is
constructed), or when it exhausts all the edges (in this case, the vertices s
and t are disconnected).
Given the leveled graph L, we nd all paths in L from the source s to
the sink t as follows. Starting from the vertex s, we follow the edges of L
to nd a path p of length level[t]. Since the graph L is leveled, the path p

9
can be found in a straightforward way (i.e., at each vertex, simply follow
an arbitrary edge from the vertex). Thus, the path p can be constructed in
time O(level[t]) = O(n), where n is the number of vertices in G. Now if the
ending vertex is t, then we have found a path from s to t. We trace back
the path p to nd the edge e on p with minimum capacity c. Now we can
push c amount of ow along the path p. Note that this cuts at least one
edge, e.g. the edge e, from the path p. On the other hand, if the ending
vertex v of p is not t, then v must be a "deadend". Thus, we can cut all
incoming edges to v . In conclusion, in the above process of time O(n), at
least one edge is removed from the leveled graph L. Thus, after at most e
such processes, the vertices s and t are disconnected, i.e., all shortest paths
from s to t are saturated. This totally takes time O(ne). We give a formal
description for the above process.
Algorithm 4.2 SATURATING
Input: Leveled graph L.
1. while there is an edge from s do
find a path p of maximal length from s
if p leads to t
then saturate p and delete at least one edge on p.
else delete the last edge on p.
Now the complete version for Dinic's algorithm can be given as follows.
Algorithm 4.3 Max-Flow Dinic
Input: A flow-graph . G
Output: A maximum flow f
on . G
1. Let ( f u; v
) = 0 for all vertex pairs ( ); u; v
2. Construct the residual graph f ; G
3. whilethere is a positive flow in f G do
Call BFS Dinic on f toGconstruct the leveled graph ; L
Call SATURATING to saturate all paths in ; L
Let  be the flow in
f G
f constructed by SATURATING;
Let f f f 
= + be the new flow in ; G
Construct the residual graph f ; G
By the above discussion, each execution of the body of the while loop
in Algorithm 4.3 takes time O(ne). Now we study the number of times the
body of the while loop is executed.

10
Theorem 4.1 On a ow-graph G of n vertices, the body of the while loop
in Step 3 of Algorithm 4.3 is executed at most n 1 times.
proof. We rst prove that after each execution of the body of the while
loop, the length of the shortest path in the ow-graph is increased by at
least one. We need some notations. Let G be a ow-graph, let f be the ow
obtained by one execution of the body of the while loop on the ow-graph
G, and let Gf be the residual graph of G on the ow f . For any vertex v of
G, let level(v ) be the distance from s to v in the graph G, and let levelf (v )
be the distance from s to v in the graph Gf .
Claim 1: Suppose (v; w) is an edge in Gf , then level(w)  level(v) + 1.
Proof for Claim 1: (v; w) can be an edge in Gf due to two cases:
Case 1: (v; w) is an edge in G. Then either the vertex w is seen before
we start the search from the vertex v | in this case the level of w cannot
be larger then level(v ) + 1, or the vertex w is discovered in the search from
v | in this case, the level of w is exactly one plus the level of v .
Case 2: (v; w) is not an edge in G. Since (v; w) is an edge in the residual
graph Gf of G on the ow f , we must have that (w; v ) is an edge in G and
there is a positive ow in f from the vertex w to the vertex v . Since we only
push ow in the leveled graph L on edges that only connect consecutive levels
of vertices, we conclude that level(v ) is one plus level(w). Thus, certainly
we also have level(w)  level(v ) + 1.
Claim 2: For all vertices v, we have level(v)  levelf (v).
Proof for Claim 2: Let r = level f (v ) be the distance from s to v in the graph
Gf . Let (s; x1; x2; : : :; xr 1 ; v ) be a shortest path in Gf from s to v . Then
level(v )  level(xr 1 ) + 1 fdue to Claim1g
 level(xr 2) + 2




 level(x1) + (r 1)
 level(s) + r
= r = level f (v )
In particular, we have level(t)  levelf (t), which implies that the length
of the shortest path from s to t is not decreased after each execution of the
body of the while loop.

11
Claim 3: level(t) < levelf (t).
Proof for Claim 3: It has been already shown that level(t)  levelf (t) in
Claim 2. Hence, to prove Claim 3, we only need to show that level(t) and
levelf (t) are dierent. Let us assume the contrary that level(t) = levelf (t) =
r and derive a contradiction.
Let P = (s; x1; x2; : : :; xr 1 ; t) be a shortest path in the graph Gf from
the source s to the sink t. Then we must have
level f (t) = levelf (xr 1 ) + 1 = : : : = levelf (s) + r = r
By Claim 1, we have
level(t)  level(xr 1) + 1
 level(xr 2) + 2




 level(x1) + (r 1)
 level(s) + r
= r
By our assumption, we also have level(t) = r, thus all inequalities \" in the
above formula should be equality \=". This gives level(xi+1) = level(xi )+1
for all i = 1; : : :; r 2, level(x1 ) = level(s)+1, and level(t) = level(xr 1 )+1.
Now we show that P is also a path in the graph G. In fact, if (s; x1) is not
an edge in G, then since (s; x1) is an edge in Gf , (x1; s) must be an edge in
G and we have pushed a ow in f along the edge (x1; s). But this implies
that (x1; s) is an edge in the leveled graph L so level(x1 ) + 1 = level(s),
contradicting the fact that level(x1) = level(s) + 1, Thus, (s; x1) is an edge
in G. Similarly, all edges on the path P are edges in the graph G. Therefore,
the path P is also a path in the graph G. Since the length of the path P is
r = level(t), P is a shortest path in G. By our SATURATING algorithm,
at least one of the edges on P is saturated, thus at least one of the edges
on P should not appear in the residual graph Gf . This contradicts the
assumption that P is also a path in the graph Gf . The contradiction proves
level(t) < levelf (t).
Thus, each execution of the body of the while loop in Algorithm 4.3
increases the length of the shortest path from s to t in the ow graph Gf
by at least 1.
Now we can complete the proof of the theorem. Since we start with the
original ow-graph G in which the length of the shortest paths from s to
12
t is at least one (we can always assume that the source s and the sink t
are dierent), if the body of the while loop were executed more than n 1
times, Claim 3 says that the length of the shortest path from s to t in the
resulting residual graph Gf would be at least n, i.e., would consist of more
than n vertices. But this contradicts the fact that the graph Gf has only n
vertices.
Theorem 4.2 The running time of Dinic's Maximum Flow algorithm (Al-
gorithm 4.3) is O(n2 e).

13
 CPSC-669 Computational Optimization
Lecture #5, September 8, 1995
Lecturer: Professor Jianer Chen
Scribe: Weijie Zhang
Revision: Jianer Chen
5 Max-Flow Problem (Contd.)
5.1 Edmonds-Karp's Algorithm
We rst give a formal proof for a claim we made in the last lecture. Recall
that we denote by level(v ) and levelf (v ) the distance from the source node
s to the node v in the ow-graphs G and Gf , respectively.
Lemma 5.1 Let G be a ow-graph and let f be the ow generated by an
execution of the body of the while loop in Dinic's algorithm. If (u; v ) is an
edge in the residual graph Gf and level(u) = level(v ) 1 in G, then (u; v )
is also an edge in the original ow-graph graph G.
proof. Suppose (u; v ) is not an edge in G. Since (u; v ) is an edge in the
residual graph Gf , we must have that (v; u) is an edge in G and we pushed
a ow in f from vertex v to vertex u. However, since each execution of
the while of Dinic's algorithm pushes ow only in the leveled graph L, we
conclude that
level(v ) + 1 = level(u)
This contradicts the condition given in the lemma that level(u) = level(v ) 1.

In the last lecture, we have proved that if level f (t) = level(t), then for
a shortest path P = (s; x1; : : :; xr 1 ; t) in the graph Gf , we must have
level(xi) = level(xi+1 ) 1, level(s) = level(x1 ) 1, and level(xr 1 ) =
level(t) 1 in G. Applying Lemma 5.1 claims that P is also a path in the
original graph G. Since P is a shortest path in Gf and level(t) = level f (t), P
is also a shortest path in the original graph G. Consequently, P is contained
in the leveled graph L. By the subroutine SATURATING, all paths in the
leveled graph L are saturated. Thus, the path P in G should have also been
saturated, and at least one of the edges on P should have not appeared in

14
the residual graph Gf . But this contradicts the assumption that P is a path
in Gf . This contradiction combined with the inequality level f (t)  level(t)
gives
levelf (t) > level(t)
Therefore, each execution of the body of the while loop in Dinic's al-
gorithm (Algorithm 4.3) increases the length of the shortest path in the
ow-graph Gf by at least 1. Since the lengths of the shortest paths in Gf
cannot be larger than n 1, the while loop can be executed at most n 1
times. Moreover, as we have discussed before, each execution of the body
of the while loop takes time O(ne). This concludes that Dinic's algorithm
runs in time O(n2 e).
It will be interesting to compare Dinic's algorithm with Edmonds-Karp's
algorithm, which also uses the strategy of nding shortest augmenting path.
Instead of nding all shortest paths, Edmonds-Karp's algorithm nds just
one shortest path each time and saturates the path. The algorithm can be
given as follows.
Algorithm 5.1 Edmonds-Karp
Input: a flow-graph G
Output: a maximum flow on G
1. let f be the zero flow;
2. construct the residual graph f ; G
3. while there is a positive capacity path in f P G do
find a shortest positive capacity path 0 ; P
increase the flow f P
along the 0 as much as possible;
G
construct f for the new ; f
We omit the detailed analysis here. An informal analysis can be given as
follows. Finding a single shortest path from s to t can be done using breadth
rst search in time O(e). Other steps in the loop can easily be done in time
O(e). Thus, each execution of the body of the while loop takes time O(e).
Each execution of the body of the while loop in the above algorithm cuts at
least one edge from a shortest path. Therefore, after at most e executions,
all shortest paths of the same length have been cut so that the length of
the shortest paths in the ow-graph Gf must be increased by at least 1.
Now using the same argument as above, the length of the shortest paths
cannot be larger than n 1. Therefore, after at most O(en) executions of
the body of the while loop in the above algorithm, there will be no positive

15
capacity path from s to t in Gf and the algorithm stops with a maximum
ow. This concludes that Edmonds-Karp's algorithm runs in time O(ne2 ),
which is slightly worse than Dinic's algorithm.
5.2 Multiple source-sink ow problem
We say that a ow-graph G is a multiple source-sink ow-graph if G has
more than one source or more than one sink (or both). The multiple source-
sink ow problem can be reduced to the single source-sink ow problem as
follows.
1. add a new source S and add a new sink T ;
2. add directed edges which goes from the new source S to all old sources
in the original ow-graph, and add an directed edge from every old sink
to the new sink T .
3. dene the capacity of every new added edge. We can simply let the
capacity be a very large number. For example, this number can be the
sum of the capacities of all edges in the original ow graph.
This is easy to see that a maximum ow in the new constructed single
source-sink ow-graph gives a maximum ow in the original multiple source-
sink ow-graph.
5.3 Graph Matching
Denition 5.1 Given an undirected graph
0
G = (V; E ), a maximum match-0
ing is a maximum subset of edges E of E such that no two edges in E
share a common endpoint.
Using the formal denition of an optimization problem, we can formulate
the Graph Matching problem as a 4-tuple Q = (IQ ; SQ; fQ; optQ ), where:
IQ: the set of all undirected graphs G = (V; E );
SQ: given G = (V; E ) 2 IQ, SQ(G) is the collection of all subsets E 0 of E
such that no two edges in E 0 share a common endpoint;
fQ: given G 2 IQ and E 0 2 SQ(G), fQ(G; E 0) is equal to the number of
edges in E 0;

16
optQ : max
In this lecture we will discuss a special case: to nd maximum matchings
in bipartite graphs.
Denition 5.2 A bipartite graph is an undirected graph G = (V; E ) in
which V can be partitioned into two sets V1 and V2 such that (u; v ) 2 E
implies either u 2 V1 and v 2 V2 or u 2 V2 and v 2 V1. That is, all edges go
between the two sets V1 and V2 .
There are several approaches to solve the maximum matching problem
in bipartite graphs.
 We can use the method of augmenting paths, which is described in our
Algorithm Analysis course. The time complexity for this method is
O(ne). We will give a more detailed and careful study on this method
for general non-bipartite graphs.
 We can use Dinic's Algorithm to nd a maximum matching in an
undirected bipartite graph G = (V; E ) by constructing a ow graph
in which ows correspond to matchings. We dene the corresponding
ow graph G0 as follows:
a. add two new vertices, let them be the source s and the sink t,
b. add new directed edges from the source s to the vertices in V1
and new directed edges from the vertices in V2 to the sink t,
c. give each edge in the original graph G a direction so all these
edges go from V1 to V2,
d. assign unit capacity to each edge in the graph G0.
The proof of the following Theorem is straightforward and left for the
reader.
Theorem 5.2 A maximum matching in a bipartite graph G corresponds
directly to a maximum ow in the ow-graph G0.
If we apply Dinic's Algorithm directly to the above ow-graph G0 , we can
only claim a time bound O(n2e), which is worse than the augmenting path
method. However, a more careful analysis plus a slight modication will
show that the running p time of Dinic's Algorithm on the above ow-graph
0
G is bounded by O( ne), thus a better result than the direct augmenting
path method. The details of this analysis and the modication will be given
later in this course.
17
 CPSC-669 Computational Optimization
Lecture #6, September 11, 1995
Lecturer: Professor Jianer Chen
Scribe: Weijie Zhang
Revision: Jianer Chen
6 Karzanov's Algorithm
In this lecture, we present Karzanov's Algorithm to get a maximum ow.
This approach runs in O(n3) time, thereby an improvement upon Dinic's
Algorithm which runs in O(n2 e) time. Let us review Dinic's Algorithm
rst.
Algorithm 6.1 Dinic'sAlgorithm
Input: a flow-graph G
Output: a maximum flow on G
1. let f
be the zero flow;
2. construct the residual graph f ; G
3. while there is a positive capacity path P in Gf do
begin
3.1 find all shortest paths of positive capacity from s
to in t f G
3.2 increase the flow f
along these paths as much as
possible;
3.3 G
construct f for the new flow ; f
end
Step 3.1 can be done in O(e) time by Breadth-First Search, and Step
3.3 can easily done in time O(e). Moreover, we have already proved that
the while loop can be executed at most n 1 times. Finally, our early
implementation shows that Step 3.2 takes time O(en). Therefore if we want
to improve the time complexity from O(n2e) to O(n3), what we need to do
is to improve the running time of Step 3.2. Now the question is how to
improve it.
Let us have a closer look at our implementation of Step 3.2 in Dinic's
algorithm. With the leveled graph L being constructed, we iterate the pro-
cess of searching a path in L from the source s to the sink t, pushing ow
18
along the path, and saturating (thus cutting) at least one edge on the path.
In the worst case, for each such a path, we may only be able to cut one edge.
Therefore, to ensure that the leveled graph L is eventually cut, we may have
to perform the above iteration e times.
The basic idea of Karzanov's algorithm is to reduce the number of times
of the above iteration from e to n. In each iteration, instead of saturating
an edge in L, Karzanov saturates a vertex in L. Since there are at most n
vertices in the leveled graph L, the number of iterations is bounded by n.
Denition 6.1 Let v be a vertex in the leveled graph L = (V0; E0). Dene
the capacity cap(v ) of the vertex v to be
0 1
X X
cap(v) = min @ cap(w; v); cap(v; u)A
(w;v)2E0 (v;u)2E0
That is, cap(v ) is the maximum amount of ow we can push through the
vertex v . For the source s and the sink t, we naturally dene
X X
cap(s) = cap(s; u) and cap(t) = cap(w; t)
(s;u)2E0 (w;t)2E0
If we start from an arbitrary vertex v and try to push a ow of amount
cap(v ) through v , it may not always be possible. For example, pushing
cap(v ) = 10 units ow through a vertex v may require to push 5 units
ow along an edge (v; w), which requires that cap(w) is at least 5. But
the capacity of the vertex w may be less than 5, thus we would be blocked
at the vertex w. However, if we always pick the vertex w in L with the
smallest capacity, this problem will disappear. In fact, trying to push a ow
of amount cap(w) will require no more than cap(v ) amount of ow to go
through a vertex v for all vertex v . Therefore, we can always push the ow
all the way to the sink t (assuming we have no deadend vertices). Similarly,
we can pull this amount cap(w) of ow from the incoming edges of w all the
way back to the source s. Note that this process saturates the vertex w.
Thus, the vertex w can be removed from the leveled graph L in the rest of
the iterations of the algorithm SATURATING on L.
Now we can formally describe Karzanov's Algorithm. The rst subrou-
tine deletes all deadends in the leveled graph L and computes the capacity
for each vertex in L.
Algorithm 6.2 INITIALIZATION
19
 Input: the leveled graph L
1. Perform a depth first search on L
to delete all
vertices that are not on a path from to ; s t
2. for each vertex = v 6 s; t do
in v out v fv
[ ] = 0; [ ] = 0; [ ] = 0;
3. in s 1 out t
( )=+ ; ( )=+ ; 1
4. for each edge ( u; v do
)
in v in v cap u; v
[ ] = [ ] + ( );
out u out u cap u; v
[ ] = [ ] + ( );
5. for each vertex v do
cap(v) = minfin[v]; out[v ]g
Here, in[v ] is the sum of capacities of all incoming edges of vertex v , out[v ]
is the sum of capacities of all outgoing edges of vertex v , and f [v ] is the
amount of ow we want to push (or pull) through vertex v .
We will always start with a vertex v with the smallest cap(v ) and push
a ow of amount cap(v ) through it all the way to the sink t. This process
is similar to the breadth rst search algorithm, starting from the vertex v .
We use the array f [
] to record the amount of ow we need to push through
the corresponding vertex. f [w] = 0 implies that the vertex w has not been
seen in the breadth rst search.
Algorithm 6.3 PUSH(v)
Input: the leveled graph L
fQ is a queue used for the breadth first search. g
1. Q v; f [v] = cap(v );
2. while Q is not empty do
3. u Q; f0 = f [u];
4. while f0 > 0 do
5. let (u; w) be the next edge from u
6. if f [w] = 0 and w 6= t then Q w;
7. if cap(u; w) < f0 then
8. cut edge (u; w);
9. f [w] = f [w] + cap(u; w); f0 = f0 cap(u; w);
10. else
11. push f0 along (u; w);
12. cap(u; w) = cap(u; w) f0; f [w] = f [w] + f0 ; f0 = 0;
13. if u 6= v then cap(u) = cap(u) f0;
20
 14. if u 6= v and cap(u) = 0
then delete u from the leveled graph L.
Note that we neither change the value cap(v ) nor remove the vertex v
from the leveled graph L. This is because the vertex v will be used again in
the following PULL algorithm.
The algorithm PULL is very similar to algorithm PUSH. We start from
the vertex v and pull cap(v ) amount of ow all the way back to the source
vertex s. Note that now the breadth rst search is on the reversed directions
of the edges of the leveled graph L. This can be easily done by a reorgani-
zation of the adjacency list representation of the graph L and the process
can be done in time O(e) (this only needs to be done once for all calls to
PULL). Moreover, note that the only vertex that can be seen in both PUSH
subroutine and PULL subroutine is the vertex with the smallest capacity.
Therefore, no updating is needed for array f [
].
Algorithm 6.4 PULL(v)
Input: the leveled graph L
fQ0 is a queue used for the breadth first search. g
1. Q0 v; f [v] = cap(v );
2. while Q0 is not empty do
3. u Q0; f0 = f [u];
4. while f0 > 0 do
5. let (w; u) be the next edge into u
6. if f [w] = 0 and w 6= s then Q0 w;
7. if cap(w; u) < f0 then
8. cut edge (w; u);
9. f [w] = f [w] + cap(w; u); f0 = f0 cap(w; u);
10. else
11. push f0 along (w; u);
12. cap(w; u) = cap(w; u) f0; f [w] = f [w] + f0 ; f0 = 0;
13. cap(u) = cap(u) f0;
14. if cap(u) = 0
then delete u from the leveled graph L.
Again note that after the execution of the PULL algorithm, the vertex
v with minimum capacity always gets removed.
With the subroutines PUSH and PULL, a new saturating subroutine can
be given as follows.
21
Algorithm 6.5 SATURATING-Karzanov
Input: the leveled graph L
Output: a flow on f L
that saturates all paths in L
1. call INITIALIZATION;
2. while there is a path from sto t inL do
3. let v
be the vertex in L with minimumcap(v );
4. v
call PUSH( );
5. v
call PULL( );

We now analyze the algorithm SATURATING-Karzanov.

Lemma 6.1 The algorithm SATURATING-Karzanov takes time O(n2).
proof. Step 1 takes time O(e) = O(n2). Steps 3 takes time O(n). Since
each execution of the loop body Steps 3-5 deletes at least one vertex from L,
the while loop body (Steps 3-5) is executed at most n times. Therefore, all
executions of Step 3 in the algorithm SATURATING-Karzanov take time
O(n2).
Now we study the complexity of Steps 4 and 5. Let us rst consider
the subroutine PUSH. To push a ow of amount f [u] through a vertex u,
we take each outgoing edge from u. If the capacity of the edge is smaller
than the amount of ow we need to push, we saturate the edge, and if the
capacity of the edge is not smaller than the amount of ow we need to push,
we let all remaining ow go along that edge and jump out from the while
loop of Steps 4-12 in the algorithm PUSH. Moreover, once an edge gets cut
at Step 8 of the algorithm, the edge will never appear in the leveled graph L
for the later calls for PUSH in the while loop of Algorithm SATURATING-
Karzanov. Thus, each execution of the while loop body Steps 5-12 in the
algorithm PUSH, except the last one, deletes an edge from the leveled graph
L. Therefore, the number of total such executions cannot be larger than e.
Consequently, all such executions in the algorithm SATURATING-Karzanov
take time O(e) = O(n2 ). Besides these executions, the subroutine PUSH
spends constant time on each vertex u, thus O(n) time on the graph L. Since
there are only O(n) calls to the PUSH in the algorithm SATURATING-
Karzanov, we conclude that the algorithm SATURATING-Karzanov takes
time O(n2 ) on all calls to PUSH. Similarly, the total time spent on the calls
to PULL is also bounded by O(n2 ).
Algorithm 6.6 Karzanov's Algorithm

22
 Input: a flow-graph G
Output: a maximum flow on G
1. let f
be the zero flow;
2. construct the residual graph f ; G
3. while
there is a path from to s
in f t G do
3.1. construct the leveled graph ; L
3.2. call SATURATING-Karzanov to find a flow f to
saturate ; L
3.3. = +  ; construct f for the new flow
f f f G f;
Theorem 6.2 Karzanov's Algorithm (Algorithm 6.6) runs in time O(n3)
proof. According to the discussion of Dinic's algorithm, we know that the
body of the while loop in Algorithm 6.6 is executed at most n 1 times.
Moreover, Steps 3.1 and 3.3 takes time O(e) = O(n2 ). By Lemma 6.1,
each call to the subroutine SATURATING-Karzanov takes time O(n2). We
conclude that Karzanov's algorithm takes time O(n3 ).

23
 CPSC-669 Computational Optimization
Lecture #7, September 13, 1995
Lecturer: Professor Jianer Chen
Scribe: Li Shao
Revision: Jianer Chen
7 Maximum matching on bipartite graphs
In this lecture, we study maximum matching problem on bipartite graphs.
We show that the problem can be reduced to a special form of the max- ow
problem, for which Dinic's algorithm runs very eciently.
7.1 Max-Flow Min-Cut Theorem
Denition 7.1 Let G = (V; E ) be a ow graph with source s and sink t.
A partition of V = V1 [ V2 (i.e. V1 [ V2 = V and V1 \ V2 = ) is a cut if
s 2 V1, t 2 V2.
Denition 7.2 The capacity of a cut (V1,V2) is dened by the value:
X
cap(V1; V2) = cap(v; w)
v2V1 ;w2V2
The following lemma will be used in our later discussion.
Lemma 7.1 Let G = (V; E ) be a ow graph and let (V1; V2) be a cut of G.
Then for any ow f on G we have
X
jf j = f (v; w)
v2V1 ;w2V2
proof. By denition,
P
we have jf j = w2V f (s; w). By the denition of a
P
ow, we have w2V f (v; w) = 0 for all vertices v 2 V1 fsg. Therefore, we
have
X X
jf j = f (s; w) = f (v; w)
w2V v2V1 ;w2V
X X
= f (v; w) + f (v; w)
v2V1 ;w2V1 v2V1;w2V2
Now since f (v; w) = f (w; v ) for all vertices v; w 2 V1 , the rst term in the
last expression of the above equation is equal to 0. The lemma follows.
24
 Lemma 7.1 implies one direction of the following fundamental theorem
in the study of maximum ow problem.
Theorem 7.2 (Max-Flow Min-Cut Theorem) For any ow graph G =
(V; E ),
maxfjf j : f is a ow on Gg = minfcap(V1; V2) : (V1; V2) is a cut of Gg

proof. Let f be a ow on G and let (V1; V2) be a cut of G. By Lemma 7.1

and note that f is a ow on G thus f (v; w)  cap(v; w) for all vertices v 2 V1
and w 2 V2, we have
X X
jf j = f (v; w)  cap(v; w) = cap(V1; V2)
v2V1 ;w2V2 v2V1 ;w2V2
Since f is an arbitrary ow on G and (V1; V2) is an arbitrary cut of G, we
conclude
maxfjf j : f is a ow on Gg  minfcap(V1; V2) : (V1; V2) is a cut of Gg
To prove the other direction, let f be a maximum ow on the ow graph
G. Let Gf be the residual graph of G with respect to f . By Theorem 3.1,
there is no path from the source s to the sink t in the residual graph Gf .
Dene V1 to be the set of vertices that are reachable from the source s in
the graph Gf . Thus, s 2 V1 and t 62 V1. Therefore, if we let V2 = V V1,
then (V1; V2) is a cut of the ow graph G. Now let e = (v; w) be an edge in
G such that v 2 V1 and w 2 V2. Since e is not an edge in the residual graph
Gf (otherwise, the vertex w would be reachable from s in Gf ), the edge e
must be saturated by the ow f . That is, f (v; w) = cap(v; w). Therefore,
we have
X X
cap(V1; V2) = cap(v; w) = f (v; w)
v2V1 ;w2V2 v2V1 ;w2V2
By Lemma 7.1, the last expression is equal to jf j. This proves
maxfjf j : f is a ow on Gg  minfcap(V1; V2) : (V1; V2) is a cut of Gg
The proof of the theorem is thus completed.

25
7.2 Hopcroft and Karp's analysis
Now we describe an analysis given rst by Hopcroft and Karp, which gives an
O(pne) time algorithm for maximum matching on bipartite graphs, which
is the best algorithm known so far for the problem.
Given a bipartite graph B = (V1; V2), we can construct a ow-graph G
by adding two vertices s and t, adding a directed edge from s to each of
the vertices in V1, adding a directed edge from each of the vertices in V2 to
t, giving each original edge in B a direction from V1 to V2, and setting the
capacity of each edge in G to 1. The resulting ow-graph G has some very
interesting properties that can be characterized as follows.
Denition 7.3 A ow graph G is a simple ow-graph if it satises the
following two conditions:
1. the capacity of each edge of G is 1; and
2. every vertex v 6= s; t either has only one incoming edge or has only
one outgoing edge.
Clearly, the ow-graph G constructed above from a bipartite graph is a
simple ow-graph. Now consider Dinic's algorithm on a simple ow-graph
G.
Algorithm 7.1 Dinic's Algorithm

1. f = 0;
2. Construct f ;G
3. while there is a path from to ins t Gf do
construct the leveled graph ; L
saturate all the paths in from L s t
to ;
update the flow ; f
construct the new f; G
Lemma 7.3 Let G = (V; E ) be a simple ow-graph and let f be a ow on G
such that f (v; w) is either 1 or 0 for all pairs (v; w) of vertices in G. Then
the residual graph Gf is also a simple ow-graph.
proof. Consider any vertex w in G, w 6= s; t. Suppose that the vertex w
has only one incoming edge e = (v; w).
If f (v; w) = 0 then f (w; u) = 0 for all u 2 V . Thus, in the residual
graph Gf , e is still the only incoming edge for the vertex w.

26
 If f (v; w) = 1 then there must be an outgoing edge (w; u) of w such
that f (w; u) = 1, and for all other outgoing edges (w; u0) we must have
f (w; u0) = 0. Therefore, in the residual graph Gf , the edge (v; w) disappears
and we add another outgoing edge (w; v ), and the edge (w; u) disappears and
we add a new incoming edge (u; w), which is the unique incoming edge of
the vertex w in Gf .
The case that the vertex w has only one outgoing edge can be proved
similarly.
Lemma 7.4 Let G = (V; E ) be a simple ow-graph, and let f be a max-
ow on G, let l be the length of the shortest path from s to t in G. then
l  n=jf j + 1, where n is the number of vertices in G.
proof. Dene Vi to be the set of vertices of distance i from s in G.
Fix an i, 0  i  l 1. Dene C1 and C2 by
[i
C1 = Vj and C2 = V C1
j =0
It is clear that (C1; C2) is a cut of the ow graph G.
We claim that for any edge e = (v; w) of G such that v 2 C1 and w 2 C2,
we have v 2 Vi and w 2 Vi+1 . In fact, if v 2 Vh for some h < i, then the
distance from s to w cannot be larger than h + 1  i. This would imply
that w is in C1. Thus, v must be in Vi. Now since e = (v; w) is an edge in
G, w is in Vk for some k  i + 1, and v is in Vi, we must have w 2 Vi+1.
This observation together with Lemma 7.1 gives us
X X
jf j = f (v; w)  f (v; w)
v2C1 ;w2C2 v2Vi;w2Vi+1
Now since G is a simple ow graph, there is at most one unit ow through
a vertex v 6= s; t. Therefore, if i = 0 (i.e., Vi = fsg), then jf j  jVi+1 j, and
if i = l + 1 (i.e., Vi+1 = ftg), then jf j  jVi j, and for 0 < i < l + 1, we have
both jf j  jVi+1 j and jf j  jVi j. Summarizing these inequalities for all i, we
get
n = jV j  jV1j + jV2j +


+ jVl 1 j  (l 1)jf j
which gives immediately l  n=jf j + 1.
Now we are ready for analyzing the complexity of Dinic's algorithm on
simple ow-graphs.
27
Lemma 7.5 For simple ow-graphs, the constructed leveled graph L in
Dinic's algorithm can be saturated in time O(e).
proof. The saturating is based on a depth rst search process, starting
from the source s. Any subtree constructed during the depth rst search
can be entirely deleted if it does not lead to the sink t. Moreover, once a
path from s to t is found, all edges on the path will be saturated because
all edges in a simple graph have capacity 1. Therefore, in this process, each
edge is processed at most twice then will be deleted from the leveled graph
L. This concludes that the running time of the saturating process can be
done in time O(e).
Since other steps in the while loop body of Dinic's algorithm can be
easily done in time O(e), we conclude that each execution of the while loop
body of Dinic's algorithm takes time O(e).
Lemma 7.6 On a simple ow-graph,
p the while loop body of Dinic's algo-
rithm is executed at most 2 n + 1 times, where n is the number of vertices
in the simple ow-graph.
proof. Let h be the number of times the while loop body of Dinic's
algorithm is executed on a simple ow graph G of n vertices. Let fmax be a
maximum ow on p G.
If jfmaxj  2 n, then of course the loop body is executed at most 2 n
p
times since each execution of the loop body increases the ow value by at
least 1. p
Now assume jfmaxj > 2 n. Let k0 be the largest integer such that after
k0 executions of the while loop body,
p the ow f0 constructed in Dinic's
algorithm is still less than jfmaxj n. A few interesting facts about k0 are
 k0 < h;
 after (k0 +1)st execution of the while loop
p body in Dinic's algorithm,
the constructed ow is at least jfmaxj n;
 the
pn. value of the maximum ow in the ow graph Gf0 is larger than

p path from s to t in the ow

By the third fact, the lengthpof the shortest
graph Gf0 is bounded by n= n + 1 = n + 1. Now since each execution
of the while loop body increases the length of the shortest path from s to
28
t by atp least 1 (see Claim 3 in the proof of Theorem 4.1), we conclude that
k0  n.
By the second fact, after (k0 + 1)st execution of the while loop
pn. body in
Dinic's algorithm, pthe constructed ow f1 is at least jfmaxj There-
fore, with another n executions of the while loop body, starting from the
ow-graph Gf1 , Dinic's algorithm must reach the maximum ow value fmax
because each execution of the while loop body increases the ow value by
at least 1. p p
In conclusion, we have h  k0 + 1 + n  2 n + 1. This completes the
proof.
Theorem 7.7 Dinic's algorithm runs in time O(pne) on a simple ow
graph of n vertices and e edges.
proof. Follows directly from Lemma 7.5 and Lemma 7.6.
Corollary 7.8 The
p maximum matching problem on bipartite graphs can be
solved in time O( ne).

29
 CPSC-669 Computational Optimization
Lecture #8, September 15, 1995
Lecturer: Professor Jianer Chen
Scribe: Li Shao
Revision: Jianer Chen
8 Maximum matching for general graphs
Now we study the maximum matching problem on general graphs. Recall
that a matching M on a graph G = (V; E ) is a subset of edges in E such that
no two edges in M share a common endpoint. A vertex v is a matched vertex
if v is an endpoint of an edge in M , otherwise, the vertex is an unmatched
vertex.
Denition 8.1 Let M be a matching in a graph G. An alternating path is
a simple path p = fu0; u1; u2; : : :g such that the vertex u1 is unmatched and
that the edges (u2i 1; u2i) are in M , for i = 1; 2; : : :. An alternating path is
an augmenting path if it starts and ends with unmatched vertices.
Note that alternating paths and augmenting paths are relative to a xed
matching M . The following theorem serves as a fundamental theorem in
graph matching.
Theorem 8.1 Let G be a graph and let M be a matching in G. M is
maximum if and only if there is no augmenting path in G.
proof. Suppose that there is an augmenting path p = (u0; u1; : : :; ur ) in
the graph G with respect to the matching M .
It is easy to see that the length r of p is odd. Let r = 2h + 1, where h is
an integer. Consider the set of edges M 0 = M  p, where  is the symmetric
dierence dened by A  B = (A B ) [ (B A). Since the number of edges
on p that are in M is one less than the number of edges on p that are not in
M , the number of edges in M 0 is one more than that in M . It is also easy
to check that M 0 is also a matching in G: M 0 = M  p = (M p) [ (p M ),
for any two edges e1 and e2 in M 0 , (1) if both e1 and e2 are in M p then
they are in M so have no common endpoint because M is a matching; (2)
if both e1 and e2 are in p M then e1 and e2 have no common endpoint

30
because p is alternating; and (3) if e1 is in M p and e2 is in p M then e1
cannot have an endpoint on p since the two endpoints of p are unmatched
and all other vertices on p are matched by edges on p.
Therefore, M 0 is a matching larger than the matching M . This proves
that if there is an augmenting path p, then the matching M cannot be
maximum.
Conversely, suppose that the matching M is not maximum. Let Mmax
be a maximum matching. Then jMmaxj > jM j. Consider the graph G0 =
Mmax  M = (M Mmax) [ (Mmax M ). No vertex in G0 has degree larger
than 2. In fact, if a vertex v in G0 had degree larger than 2, then at least
two edges incident on v belong to either M or Mmax, contradicting the fact
that both M and Mmax are matchings in G. Therefore, each component of
G0 must be either a simple path, or a simple cycle. In each simple cycle
in G0 , the number of edges in Mmax M should be exactly the same as
the number of edges in M Mmax. For each simple path in G0 , either
the number of edges in M Mmax is the same as the number of edges in
Mmax M (in this case, the path has an even length), or the number of
edges in M Mmax is one more than the number of edges in Mmax M , or
the number of edges in Mmax M is one more than the number of edges in
M Mmax. Since jMmaxj > jM j, we conclude that there is at least one path
p = (u1; u2; : : :; u2h+1) in G0 in which the number of edges in Mmax M is
one more than the number of edges in M Mmax. Note that the endpoint u1
of the path p must be unmatched in M . In fact, since (u1 ; u2) 2 Mmax M ,
if u1 is matched in M by an edge e, we must have e 6= (u1 ; u2). Now since u1
has degree 1 in G0, e 62 G0, e is also contained in Mmax. This would make
the vertex u1 incident on two edges (u1 ; u2) and e in the matching Mmax.
Similar reasoning shows that the vertex u2h+1 is also unmatched in M . In
consequence, the path p is an augmenting path in the graph G with respect
to the matching M .
This completes the proof.
Based on the above theorem, a maximum matching algorithm can be
given as follows.
Algorithm 8.1 Max-matching for general graphs

1. M = ;
2. while there is an augumenting path in G do
find an augmenting path ; p
construct the matching = M M p
with one more edge;

31
 Since a matching in a graph G of n vertices cannot contain more than
n=2 edges, the while loop in the above algorithm will be executed at most
n=2 times. In the next lectures, we will show how an augmenting path can
be constructed when a matching is given for a graph.

32
 CPSC-669 Computational Optimization
Lecture #9, September 18,1995
Lecture: Professor Jianer Chen
Scribe: Shijin Lu
Revision: Jianer Chen
9 Theorems on maximum matching problem
Let us rst review the fundamental theorem and algorithm for maximum
matching for general graphs.
Theorem 9.1 Let G be a graph and let M be a matching in G. M is
maximum if and only if there is no augmenting path in G.
Algorithm 9.1 Max-matching for general graphs

1. M = ;
2. while there is an augumenting path in G do
find an augmenting path ; p
Let M M p
= ;

All known algorithms for maximum matching of general graphs are based
on Theorem 9.1 and Algorithm 9.1. The main point here is how an aug-
menting path can be found. For the rest of the discussion, we assume that
G is a xed graph and that M is a xed matching in G.
Observe that an augmenting path P must start with an unmatched ver-
tex v0 . The next vertex v1 must be a neighbor of v0. If v0 is also unmatched,
then the edge (v0 ; v1) constitutes an augmenting path. On the other hand,
if the length of P is larger than 1, then the third vertex v2 on P must be the
one that matches v1 in M . Now since the path fv0; v1; v2g does not make
an augmenting path, the fourth vertex v3 must be a neighbor of the vertex
v2 , and so on. Therefore, it seems that we can search the augmenting path
using a breadth rst search manner: start with v0 , then search all neighbors
of v0 , then search all vertices that match the neighbors of v0 , and so on until
we nd an unmatched vertex. In this search, we give each vertex v a level
number level[v ] such that all roots of the breadth rst search trees are at
level 0, and that the children of a vertex at level i are at level i +1. A vertex
will be called an even level vertex or an odd level vertex according to its level
33
number. On each even level vertex v2h , we search all neighbors of v2h , and
on each odd level vertex v2h+1 , we only take the unique vertex v2h+2 such
that the edge (v2h+1 ; v2h+2 ) is in the matching M .
Another modication we will made is that we will perform this BFS
fashion search starting from all unmatched vertices at the same time, instead
of starting from a single vertex. Implementation of this modication is
simple: as for the standard BFS, we use a rst-in rst-out queue. However,
we rst put all unmatched vertices in the queue then perform the BFS
fashion search until either an augmenting path is found or the queue Q is
empty. It is easy to see that this search will rst construct the rst level for
all BFS trees rooted at the unmatched vertices, then the second level for all
BFS trees, and so on.
Remark 9.1 These modications make the BFS trees lose many of their
well-known and nice properties. The following lost properties should be
mentioned:
(1) in the modied BFS process, a cross-edge (i.e., an edge of G that links
two vertices in the BFS trees that do not have a father-son relation) may
link two vertices whose level numbers dier by an arbitrarily large number.
On the other hand, in the standard BFS process, each cross-edge links two
vertices whose level numbers dier by at most 1;
(2) in the modied BFS process, a tree path from an ancestor to a
descendent may no longer be a shortest path between the two vertices; and
(3) in the modied BFS process, a tree may not necessarily contain all
vertices in a connected component of the graph G. In fact, now there may
be cross-edges that link two vertices in two dierent BFS trees.
We present our rst draft of the algorithm.
Algorithm 9.2 Modified BFS (Version 1)

1. put all unmatched vertices in the queue ; Q

2. while no augmenting path has been found do
Let v be the next vertex in the queue ; Q
if v is an even level vertex
then make all unvisited neighbors of the children v
of vand add them to Q
else fv is an odd level vertex. g
if the vertex w
that matches v
is unvisited,
then make w
a child of and addv to . w Q
34
 Algorithm 9.2 does not describe the details of how an augmenting path
can be found, which is discussed in the rest of this section.
Denition 9.2 In the modied BFS process, a cross-edge e is a good cross-
edge if either e 2 M and e links two odd level vertices in two dierent BFS
trees, or e 62 M and e links two even level vertices in two dierent BFS trees.
Lemma 9.2 If a good cross-edge is given in the modied BFS process, then
an augmenting path can be constructed in linear time.
proof. Let e = (v2s+1 ; u2t+1) be a good cross-edge such that e 2 M ,
fv0; v1; : : :; v2s+1g is a tree path in a BFS tree Tv from root v0 to v2s+1, and
fu0; u1; : : :; u2t+1g is a tree path in a BFS tree Tu from root u0 to u2t+1,
where v0 = 6 u0. By the modied BFS process, v0 and u0 are unmatched
vertices, the edges (v2i ; v2i+1) and (u2j ; u2j +1) are not in M , for all i =
0; : : :; s and j = 0; : : :; t, and the edges (v2i+1 ; v2i+2) and (u2j +1; u2j +2 ) are
in M , for all i = 0; : : :; s 1 and j = 0; : : :; t 1. Therefore, the path
fv0; v1; : : :; v2s; v2s+1; u2t+1; u2t; : : :; u1; u0g
is an augmenting path that can be easily constructed in linear time.
The case that the good cross-edge is not in M and links two even level
vertices can be proved similarly.
Lemma 9.2 suggests a renement of Algorithm 9.2.
Algorithm 9.3 Modified BFS (Version 2)

1. put all unmatched vertices in the queue ; Q

2. while no augmenting path has been found do
Let v be the next vertex in the queue ; Q
if v is an even level vertex then
for each neighbor of w v do
if v; w
( ) is a good cross-edge then
construct an augmenting path and stop;
if w is unvisited then
make w a child of and add vto ; w Q
else fv is an odd level vertex. g
let w be the vertex matching ; v
if v; w
( ) is a good cross-edge then
35
 sv0 u0 s sv0 u0 s
s sv1 u1 s s sv1 u1 s
sv2 u2 s sv2 u2 s
QQQ
v6s  QQ v6
s s

v3 
s v3 
s

Qs
s 4 v v
s 5 sv4 sv5

s SSs

v v0
s SSs
  S
v

s Ss s Ss S
v0
s s s s

(a) (b)
Figure 1: The structure of a blossom
construct an augmenting path and stop;
if w is unvisited then
make wa child of v
and add to ; w Q
for years people believed that Algorithm 9.3 was sucient for construct-
ing an augmenting path until the following structure was discovered.
Denition 9.3 In the modied BFS process, a cross-edge e is a bad cross-
edge if either e 2 M and e links two odd level vertices in the same BFS tree,
or e 62 M and e links two even level vertices in the same BFS tree.
Let us consider how a bad cross-edge may make the modied BFS process
fail to nd an existing augmenting path. Let e = fv; v 0g be an edge in M
such that both v and v 0 are odd level vertices in the same BFS tree. When
we rst encounter the edge e in the modied BFS process, both vertices v
and v 0 have been visited. Therefore, the edge cannot be added to the BFS
tree. However, the edge cannot be simply ignored since it may \hide" an
augmenting path from our BFS process. Consider the case in Figure 1(a),
where each single line represents an edge not in the matching M and each
double line represents a matched edge in M .
The only two unmatched vertices in Figure 1(a) are v0 and u0 . Thus, the
modied BFS process starts with v0 and u0 being in the queue Q and stops
as in Figure 1(b) without nding an augmenting path. Note that the edge
(v6 ; u2) is not included in the BFS trees because at time u2 is expanding, v6
has been visited through v2 , and at time v6 is expanding, only the matched
36
edge (v6 ; v5) is considered. The edge (v; v 0) is also not included in the BFS
trees because at time v (resp. v 0 ) is processed, v 0 (resp. v ) has been visited.
However, there is clearly an augmenting path from v0 to u0:
fv0; v1; v2; v3; v4; v; v0; v5; v6; u2; u1; u0g
In a similar case when e is a bad edge such that e is not in M and e
links two even level vertices in the same BFS tree, we can also construct
a conguration in which the modied BFS process fails to nd an existing
augmenting path.
This discussion motivates the following denition.
Denition 9.4 In the modied BFS process, a blossom is a simple cycle
consisting of a bad cross-edge e = (v; v 0) together with the two unique tree
paths from v and v 0 to their least common ancestor v 00. The vertex v 00 will
be called the base of the blossom.
For example, the cycle fv2 ; v3; v4; v; v 0; v5; v6; v2g is a blossom whose base
is v2 .
Remark 9.5 There are a number of interesting properties for blossoms.
We list those that are related to our later discussion.
(1). A blossom consists of an odd number of vertices. This is because
either both ends v and v 0 of the bad cross-edge are odd level vertices or both
v and v0 are even level vertices.
(2). Suppose that the cycle b = fv0 ; v1; : : :; v2s ; v0g is a blossom, where
v0 is the base, then the edges (v2s ; v0) and (v2i; v2i+1) for all i = 0; : : :; s 1,
are not in the matching M , and the edges (v2j 1 ; v2j ) for all j = 1; : : :; s are
in the matching M .
(3). If an edge e0 is not contained in a blossom but is incident to a vertex
v in the blossom, then the edge e0 cannot be in the matching M unless the
incident vertex v is the base of the blossom. This is because each vertex,
except the base, in a blossom is incident on a matched edge in the blossom.
Identifying and constructing a blossom is easy, as stated in the following
lemma.
Lemma 9.3 In linear time, we can identify a bad cross-edge and construct
the corresponding blossom.
proof. In the modied BFS process, we keep track of the level number
37
for each visited vertex. Once a cross-edge e = (v; v 0) is found, we can follow
the tree edges back to the root to check whether the two ends v and v 0 of e
belong to the same BFS tree. This together with the level numbers of v and
v 0 is sucient for deciding if e is a bad cross-edge. For a bad cross-edge, we
trace the two tree paths back from the common root to nd the last common
vertex v 00 on the paths. The vertex v 00 is the base for the blossom.
Thus, blossoms are structures that may make the modied BFS process
fail. Is there any other structure that can also fool the modied BFS process?
Fortunately, blossoms are the only such structures, as we will discuss below.
We start with the following lemma.
Lemma 9.4 If a matched edge in M is a cross-edge, then it is either a good
cross-edge or a bad cross-edge.
proof. Let e = (v; v 0) be a matched edge that is a cross-edge. The vertices
v and v 0 cannot be roots of the BFS trees since roots of the BFS trees are
unmatched vertices. Let w and w0 be the fathers of v and v 0, respectively.
The tree edges (w; v ) and (w0; v 0) are not matched edges since (v; v 0) is a
matched edge. Thus, v and v 0 must be odd level vertices. Now if v and v 0
belong to dierent BFS trees, then the edge e is a good cross-edge, otherwise
e is a bad cross-edge.
Lemma 9.5 If there is no blossom in the modied BFS process, then there
is a good cross-edge if and only if there is an augmenting path.
proof. By Lemma 9.2, if there is a good cross-edge, then there is an
augmenting path that can be constructed from the good cross-edge in linear
time.
Conversely, suppose there is an augmenting path p = fu0 ; u1; : : :; u2t+1g.
If t = 0, then the path p itself is a good cross-edge and we are done. Thus,
assume t > 0. Let v1 , : : :, vh be the roots of the BFS trees, processed in that
order by the modied BFS process. Without loss of generality, we assume
u0 = vb where b is the smallest index such that vb is an end of an augmenting
path. With this assumption, the vertex u1 is a child of u0 in the BFS tree
rooted at u0 . If any matched edge e on p is a cross-edge, then by Lemma 9.4,
e is either a good cross-edge or a bad cross-edge. Since there is no blossom,
e must be a good cross-edge again the lemma is proved.

38
 Thus, we assume that the augmenting path p has length larger than 1,
no matched edges on p are cross-edges, and u1 is a child of u0 in the BFS
tree rooted at u0.
Case 1. Suppose that all vertices on p are contained in the BFS trees.
Both u0 and u2t+1 are even level vertices. Since the path p is of odd
length, there must be an index i such that level[ui 1] = level[ui] mod 2.
Without loss of generality, assume i is the smallest index satisfying this con-
dition. The edge (ui 1 ; ui ) must be a cross-edge. Thus, by our assumption,
(ui 1 ; ui) is not a matched edge.
Suppose that both ui 1 and ui are odd level vertices, then i  2. Since
(ui 2 ; ui 1) is a matched edge, ui 2 6= u0 . Moreover, by our assumption,
(ui 2 ; ui 1) is a tree edge. Thus, ui 2 is an even level vertex. Moreover,
since (ui 2 ; ui 1 ) is a matched edge, the index i 2 is an odd number. Now
the partial path
pi 2 = fu0; u1; : : :; ui 2 g
is of odd length and has both ends being even level vertices. This implies
that there is an index j such that j  i 2 and level[uj 1] = level[uj ] mod 2.
But this contradicts the assumption that i is the smallest index satisfying
this condition.
Thus, ui 1 and ui must be even level vertices. So (ui 1 ; ui) is either a
good cross-edge or a bad cross-edge. By the assumption of the lemma, there
is no blossom. Consequently, (ui 1 ; ui ) must be a good cross-edge and the
lemma is proved for this case.
Case 2. Some vertices on p are not contained in any BFS trees.
Let ui be the vertex on p with minimum i such that ui is not contained
in any BFS trees. Then i  2.
Suppose (ui 1; ui ) 2 M . If ui 1 is an odd level vertex then ui would have
been made a child of ui 1 . Thus ui 1 is an even level vertex. However, since
ui 1 cannot be a root of a BFS tree, ui 1 would have matched its father in
the BFS tree, this contradicts the assumption that ui 1 matches ui and ui
is not contained in any BFS trees.
Thus we must have (ui 1 ; ui ) 62 M . Then (ui 2 ; ui 1) is in M . Thus,
the index i 2 is an odd number. By our assumption, (ui 2; ui 1 ) is a tree
edge. If ui 1 is an even level vertex, then ui would have been made a child
of ui 1 . Thus, ui 2 is an even level vertex. Now in the partial path of odd
length
pi 2 = fu0; u1; : : :; ui 2g;

39
all vertices are contained in the BFS trees, and the two ends are even level
vertices. Now the proof goes exactly the same as for Case 1 | we can
nd a smallest index j  i 2 such that level[uj 1] = level[uj ] mod 2 and
(uj 1 ; uj ) is a good cross-edge.
This completes the proof of the claim.
By Lemma 9.5, if there is an augmenting path and if no bad cross-edge
is found (thus no blossom is found), then the modied BFS process will
eventually nd a good cross-edge. By Lemma 9.2, an augmenting path can
be constructed in linear time from this good cross-edge. In particular, if the
graph is bipartite, then the modied BFS process will always be able to con-
struct an augmenting path if one exists, since a bipartite graph contains no
odd length cycle, thus no blossom can appear in the modied BFS process.
This gives the well-known algorithm of running time O(ne) for maximum
matching on bipartite graphs.
In order to develop an ecient algorithm for maximum matching on
general graphs, we need to resolve the problem of blossoms. Surprisingly,
the solution to this problem is not very dicult, based on the following
\blossom shrinking" technique.
Denition 9.6 Let G be a graph and M a matching in G. Let b be a
blossom found in the modied BFS process. Dene G=b to be the graph
obtained from G by \shrinking" the blossom b. That is, G=b is a graph
obtained from G by deleting all vertices (and their incident edges) of the
blossom b then adding a new vertex vb that is connected to all vertices that
are adjacent to some vertices in b in the original graph G.
It is easy to see that given the graph G and the blossom b, the graph
G=b can be constructed in linear time.
Since there is at most one matched edge that is incident to but not
contained in a blossom, for a matching M in G, the edge set M b is a
matching in the graph G=b.
Theorem 9.6 (Edmond) Let G be a graph and M a matching in G. Let b
be a blossom in G. Then there is an augmenting path in G with respect to
M if and only if there is an augmenting path in G=b with respect to M b.
proof. Suppose that the blossom is b = fv0; v1; : : :; vs ; v0g, where v0 is
the base. We rst show that the existence of an augmenting path in G=b

40
implies an augmenting path in G. Let p = fu0; u1; : : :; utg be an augmenting
path in G=b and let vb be the new vertex in G=b obtained by shrinking b.
Case 1. If the vertex vb is not on the path p, then clearly p is also an
augmenting path in G.
Case 2. Suppose vb = ut. Then vb is an unmatched vertex in the
matching M b. Consequently, the base v0 of the blossom b is unmatched
in the matching M .
If the edge (ut 1 ; ut) in G=b corresponds to the edge (ut 1 ; v0) in G, then
the path
p1 = fu0; u1; : : :; ut 1; v0g
is an augmenting path in G.
If the edge (ut 1 ; ut) in G=b corresponds to the edge (ut 1 ; vh ) in G,
where vh is not the base of b, then one of the edges (vh 1 ; vh ) and (vh ; vh+1 )
is a matched edge. Without loss of generality, suppose that (vh ; vh+1 ) is a
matched edge. Then, the path
p2 = fu0; u1; : : :; ut 1; vh ; vh+1; : : :; vs; v0g
is an augmenting path in G.
The case vb = u0 can be proved similarly.
Case 3. Suppose that vb = ud, where 0 < d < t. Then without
loss of generality, we assume that (ud 1 ; ud) is a matched edge in M b
and (ud ; ud+1 ) is an unmatched edge. The edge (ud 1 ; ud) in G=b must
correspond to the matched edge (ud 1 ; v0) in G. Let the edge (ud ; ud+1) in
G=b correspond to the edge (vh ; ud+1 ) in G.
If vh = v0 , then the path
p2 = fu0; : : :; ud 1; v0; ud+1 ; : : :; utg
is an augmenting path in G.
If vh 6= v0 , then as we proved in Case 2, we can assume that (vh 1 ; vh )
is a matched edge. Thus, the path
p3 = fu0; : : :; ud 1; v0; v1; : : :; vh 1; vh ; ud+1; : : :; utg
is an augmenting path in G.
Therefore, given an augmenting path in G=b, we are always able to con-
struct an augmenting path in G.
The proof for the other direction that the existence of an augmenting
path in G implies an augmenting path in G=b is rather complicated based
on a case by case analysis. We omit the proof here.
41
Corollary 9.7 Let G be a graph of n vertices and let b be a blossom. Given
an augmenting path in G=b, an augmenting path in G can be constructed in
time O(n).
proof. Directly follows from the construction given in the proof of The-
orem 9.6.

42
 CPSC-669 Computational Optimization
Lecture #10, September 20, 1995
Lecturer: Professor Jianer Chen
Scribe: Shijin Lu
Revision: Jianer Chen
10 Algorithms for maximum matching problem
We rst review two theorems given in the last lecture.
Theorem 10.1 Suppose we perform the modied BFS process. If there is
an augmenting path and there are no blossoms, then the BFS process will
nd an augmenting path.
Theorem 10.2 If a blossom b is found in the modied BFS process, then
there is an augmenting path in G with respect to the matching M if and only
if there is an augmenting path in G=b with respect to the matching M b.
Moreover, an augmenting path in G can be constructed in time O(n) if an
augmenting path in G=b is given.
Now the idea is fairly clear for how we can nd an augmenting path:
we perform the modied BFS process, either we nd an augmenting path
in G then we are done, or we nd a blossom then we shrink the blossom
and search an augmenting path in G=b. Once an augmenting path in G=b
is found, we can easily convert it into an augmenting path in G, as stated
in Corollary 9.7. Theorem 9.6 ensures that if we cannot nd an augmenting
path in G=b then there is no augmenting path in G.
The main algorithm for constructing a maximum matching for a general
graph now can be rewritten as follows.
Algorithm 10.1 Maximum Matching
Input: a graph ; G
Output: a maximum matching M in G
1. M = .
2. repeat
if there is an augmenting path in G w.r.t.M
then
43
 construct an augmenting path p;
letM M p
= ;
until no augmenting path is found

The process of nding an augmenting path is implemented by the mod-

ied BFS process as follows.
Algorithm 10.2 Finding An Augmenting Path

1. Perform the modified BFS process;

2. ifan augmenting path is found then
convert it to an augmenting path for the original
graph G
, stop;
3. ifa blossom is found then
construct the graph G=b
, resume the modified BFS
process;

To give a more detailed description for the modied BFS process, we

give a level number level[v ] to each vertex v . Initially, level[v ] = 1 for all
vertices v . Thus, a vertex v is visited if and only if its level number is larger
than 1.
Algorithm 10.3 Finding AnAugmenting Path (Refined)
Input: a graph G
and a matching in M G
G
Output: an augmenting path in , or report no such a path
1. for all vertices w of G do level[w] = 1;
2. for all unmatched vertices w in G do
level[w] = 0; Q w;
3. while the queue Q is not empty do
v Q;
if level[v] is even then
for each neighbor w of v do
if (v; w) is a good cross-edge then
construct an augmenting path;
convert it into an augmenting path in ; G
stop;
if v; w
( ) is a bad cross-edge then
construct the blossom b
based on ( ); v; w
construct the graph ; G=b

44
 update the queue Q
properly
go back to the beginning of Step 3;
if level[w] = 1 then
make w a child of v ;
level[w] = level[v ] + 1;
Q w;
else fv is an odd level vertex.g
let w be the vertex matching v ;
if (v; w) is a good cross-edge then
construct an augmenting path;
convert it into an augmenting path in ; G
stop;
if v; w
( ) is a bad cross-edge then
construct the blossom b
based on ( ); v; w
construct the graph ; G=b
update the queue Q
properly
go back to the beginning of Step 3;
if level[w] = 1 then
make w a child of v ;
level[w] = level[v ] + 1;
Q w;
4. fAtthis point, the modified BFS is finished without
finding an augmenting path g
return ("no augmenting path")

The correctness of Algorithm 10.3 is ensured by Lemma 9.5 and Theo-

rem 9.6.
Lemma 10.3 Algorithm 10.3 runs in time O(ne) on a graph of n vertices
and e edges.
proof. A BFS process takes time O(e).
If a blossom b is found, the graph G=b is constructed and the queue Q is
updated. It is easy to see that constructing G=b and updating Q can be done
in time O(e). Moreover, the the number of vertices in the graph G=b is at
least two less than the number of vertices in the graph G. Therefore, there
are at most n=2 blossoms found in Algorithm 10.3, and for each blossom
it takes time O(e) for Algorithm 10.3 to update the graph and the queue.
Therefore, the total time spent by Algorithm 10.3 on processing blossoms is
bounded by O(ne).
45
 Once an augmenting path is found, by Corollary 9.7, in time O(n) we can
expand a vertex back to a blossom and construct an augmenting path for the
new graph. Since there are at most n=2 such blossom restoration operations,
the total time Algorithm 10.3 spends on constructing an augmenting path
for the original graph is bounded by O(n2 ). This proves the time bound for
the algorithm stated in the lemma.
Theorem 10.4 The maximum matching problem on general graphs can be
solved in time O(n2e).
proof. By Lemma 10.3, an augmenting path can be found in time O(ne).
Since each augmenting path increases one edge for the matching, and there
are no more than n=2 edges in a matching for a graph of n vertices, the
repeat loop in Algorithm 10.1 is executed at most O(n). The theorem
follows.
We should point out that O(n2 e) is not the best upper bound for the
maximum matching problem. In fact, a moderate change in Algorithm 10.3
gives an algorithm of running time O(n3) for the problem. The basic idea
for this change is that instead of actually shrinking the blossoms, we keep
track of all vertices in a blossom by \marking" them. A careful bookkeeping
technique shows that this can be done in time O(n) per blossom. The best
known algorithm
p for the maximum matching problem on general graphs runs
in time O( ne), thus matching the best known algorithm for the maximum
matching problem on bipartite graphs.

46
 CPSC-669 Computational Optimization
Lecture #11, September 22, 1995
Lecturer: Professor Jianer Chen
Scribe: Hao Zheng
Revision: Jianer Chen
11 Linear programming problem
The Linear Programming Problem is to nd a vector (x1; x2; :::; xn) 2 Rn
such that a linear function c1x1 + c2x2 + ::: + cn xn , which is called an
objective function , is optimized (maximized or minimized) and the vector
(x2 ; x2; :::; xn) satises a given set of conditions (these conditions are called
linear constraints ).
a11x1 + a12 x2 + ::: + a1nxn  a1







ar1x1 + ar2x2 + ::: + arnxn  ar
b11x1 + b12x2 + ::: + b1nxn  b1







bs1 x1 + bs2 x2 + ::: + bsnxn  bs
d11x1 + d12x2 + ::: + d1nxn = d1







dt1 x1 + dt2x2 + ::: + dtnxn = dt
This is called the general form of Linear Programming Problem.
Using our 4-tuple formulation, the Linear Programming Problem is given
as LP = hIQ; SQ; fQ ; optQi, where
 IQ is the set of 7-tuples (c; A; B; D; a; b; d), where c = (c1; : : :; cn),
a = (a1; : : :; ar), b = (b1; : : :; bs), and d = (d1; : : :; dt) are vectors of
real numbers, A = (ai;j )rn , B = (bi;j )sn , and D = (di;j )tn are
matrices of real numbers, for some positive integers r, s, t, and n.
 for a given  = (c; A; B; D; a; b; d) 2 IQ, the solution set SQ() consists
of the set of vectors x = (x1; : : :; xn ) of real numbers that satises the
conditions Ax  a, Bx  b, and Dx = d.

47
  for a given input instance  2 IQ and a solution x 2 SQ(), the
objective function value is dened by fQ (; x) = c1 x1 +


cn xn .
 optQ is either max or min.
For many combinatorial optimization problems, the objective function
and the constraints on a solution to an input instance are linear, i.e., they can
be formulated by linear equations and linear inequalities. Therefore, optimal
solutions for these combinatorial optimization problems can be derived from
optimal solutions for the corresponding instance in Linear Programming
Problem. This is one of the main reasons why Linear Programming Problem
receives so much attention from researchers.
Example 11.1 (Maximum Flow) As an example, we show how the Max-
Flow Problem is formulated in terms of the Linear Programming Problem.
A ow-graph G of n vertices can be given by n2 non-negative real num-
bers ci;j , 1  i; j  n, where ci;j is the capacity of the edge from vertex i
to vertex j (recall that ci;j = 0 if and only if there is no edge from vertex
i to vertex j ). Here we assume that vertex 1 is the source and vertex n is
the sink. Now a ow on G (i.e., a solution to the instance G of Max-Flow
Problem) can be given by an n2 -dimensional vector
 = (f1;1; : : :; f1;n; f2;1; : : :; f2;n; : : :; fn;1; : : :; fn;n )
where fi;j is the amount of ow from vertex i to vertex j . The three condi-
tions that a ow should satisfy are trivially given by
fi;j  ci;j for 1  i; j  n
fi;j = fj;i for 1  i; j  n
X
n
fi;j = 0 i 6= 1; n
j =1
and the objective function is to maximize the linear function f1;2 + f1;3 +



+ f1;n (or equivalently, to maximize f1;n + f2;n +


+ fn 1;n ).
The standard form for Linear Programming Problem is given by
minimize c1 x1 + c2 x2 +


+ cn xn
a11x1 + a12 x2 + ::: + a1n xn = a1
a21x1 + a22 x2 + ::: + a2n xn = a2
48






(1)
am1x1 + am2 x2 + ::: + amn xn = am
x1  0; x2  0; : : :; xn  0
The general form of Linear Programming Problem can be converted into
the standard form through the following steps.
1. Converting Max to Min
Maximization maxfc1x1 + c2x2 + ::: + cn xn g can be replaced by the
equivalent condition minf( c1)x1 + ( c2)x2 +


+ ( cn )xn g.
2. Eliminating  inequalities
Each inequality bi1x1 + bi2 x2 + ::: + bin xn  bi is replaced by the
equivalent inequality ( bi1)x1 + ( bi2)x2 + ::: + ( bin )xn  ( bi ).
3. Eliminating  inequalities
Each inequality aj 1 x1 + aj 2 x2 + ::: + ajn xn  aj is replaced by the
inequality aj 1 x1 + aj 2 x2 + ::: + ajn xn yj = aj , where yj is a new
variable satisfying yj  0.
4. Eliminating unconstrained variables
For each variable xi for which xi  0 is not present, introduce two
new variables ui and vi satisfying ui  0 and vi  0, and replace the
variable xi by ui vi .
It is easy to see that the above process will convert Linear Programming
Problem from an arbitrary general form to the standard form. It is also
easy to verify that an optimal solution for the general form can be easily
derived from an optimal solution for the corresponding standard form. Thus,
we only need to concentrate on the standard form for Linear Programming
Problem.
A classical method, called Simplex Method was derived for solving Linear
Programming Problem. It is based on the following observations. Each
equation in the constraints (1) denes a hyperplane in the n-dimensional
space Rn , so the set of all points in Rn that satisfy the constraints (1)
forms a polytope in Rn , which is a convex set.1 Moreover, the objective
1 A set S in Rn is convex if for any two points x and y in S , the line segment xy is
entirely in S .

49
function c1 x1 +


+ cn xn is a convex function.2 Therefore, there is a vertex
of the polytope at which the objective function achieves its optimal value,
and this vertex can be found using greedy method. Roughly speaking, the
Simplex Method starts from an arbitrary vertex of the polytope dened by
the linear constraints (1), and uses greedy method to traverse the vertices
of the polytope until reaching a vertex at which local improvement is no
longer possible. This vertex then is an optimal solution.
In most practical cases, Simplex Method is fast enough to construct an
optimal solution for a given instance of Linear Programming Problem. It
took a while for researchers to be able to formally prove that in the worst
case, Simplex Method runs in exponential time.
It was an outstanding open problem whether Linear Programming Prob-
lem could be solved in polynomial time, until the spring of 1979, the Russian
mathematician L.G. Khachian published a proof that an algorithm, called
the Ellipsoid Algorithm, solves Linear Programming Problem in polynomial
time. Despite the great theoretical value of the Ellipsoid Algorithm, it is not
clear at all that this algorithm can be practically useful. The most obvious
among many obstacles is the large precision apparently required.
Another polynomial time algorithm for Linear Programming Problem,
called the Projective Algorithm, or more generally, the Interior Point Algo-
rithm, was published by N. Karmarkar in 1984. The Projective Algorithm,
and its derivatives, have great impact in the study of Linear Programming
Problem.

2A function f from Rn to R is convex if for any two points x and y in Rn and for any
real number 0  c  1, we have f (cx + (1 c)y)  cf (x) + (1 c)f (y).

50
 CPSC-669 Computational Optimization
Lecture #12, September 25, 1995
Lecturer: Professor Jianer Chen
Scribe: Hao Zheng
Revision: Jianer Chen
12 Integer Linear Programming Problem
Suppose that in Linear Programming Problem, we further require that
all numbers are integers, then we get Integer Linear Programming Prob-
lem. More formally, Integer Linear Programming Problem is to nd an
n-dimensional vector  = (x1 ; x2; : : :; xn) of integers such that
minimize c1 x1 + c2 x2 +


+ cn xn
a11x1 + a12 x2 + ::: + a1n xn = a1
a21x1 + a22 x2 + ::: + a2n xn = a2






(2)
am1 x1 + am2 x2 + ::: + amn xn = am
x1  0; x2  0; : : :; xn  0
where all numbers ci , aij , aj , 1  i  n, 1  j  m, are integers. The equa-
tions (2) give the standard form for Integer Linear Programming Problem.
We can similarly dene the general form for Integer Linear Programming
Problem. Moreover, it is not hard to verify that the translation steps de-
scribed in the previous Lecture Notes convert Integer Linear Programming
Problem in general form into Integer Linear Programming Problem in stan-
dard form.
It might seem that Integer Linear Programming Problem is easier since
we are working on simpler numbers. This intuition is, however, not true. In
fact, Integer Linear Programming Problem is computationally harder than
general Linear Programming Problem. This may be seen from the following
fact: now the point set dened by the constraints (2) is no longer a convex
set. It consists of discrete points in the n-dimensional Euclidean space.
Therefore, greedy algorithms based on local search do not seem to work any
more.
To formally prove the diculty of Integer Linear Programming Problem,
we need introduce some denitions.
51
Denition 12.1 An optimization problem Q1 = (I1; S1; f1; opt1) is poly-
nomial time reducible to an optimization problem Q2 = (I2; S2; f2; opt2 ) if
there are two polynomial time algorithms A1 and A2 such that (1) given an
input instance x1 2 I1 of Q1 , the algorithm A1 constructs an input instance
x2 2 I2 of Q2 , and (2) for any optimal solution y2 for the input instance
x2 of Q2, the algorithm A2 constructs an optimal solution y1 for the input
instance x1 of Q1.
The following theorem follows directly from the denition.
Theorem 12.1 Suppose that an optimization problem Q1 is polynomial
time reducible to an optimization problem Q2 , then
(1) If Q2 can be solved in polynomial time, then so can Q1 ;
(2) If Q1 cannot be solved in polynomial time, then neither can Q2 .
A problem is a decision problem if each input instance of the problem
requires only a YES/NO answer. Note that a decision problem can also be
regarded as an optimization problem in which the objective function takes
only value 0 (NO) or 1 (YES).
Recall that a decision problem is in NP if it can be solved by a nonde-
terministic algorithm running in polynomial time, and that a problem L in
NP is NP-complete if all problems in NP can be polynomial time reducible
to L. By Theorem 12.1, if any NP-complete problem is polynomial time
solvable, then P = NP. Since people commonly believe that P 6= NP, ev-
ery NP-complete problem is regarded as not solvable in polynomial time
(though there is no formal proof for this conjecture).
Denition 12.2 An optimization problem Q is NP-hard if there is an NP-
complete problem that is polynomial time reducible to Q.
According to Theorem 12.1, we accept the conjecture that every NP-hard
optimization problem is not solvable in polynomial time.
Now we are ready to show the hardness of Integer Linear Programming
Problem.
Theorem 12.2 Integer Linear Programming Problem is NP-hard.
proof. We show that the well known NP-complete problem, the Satisa-
bility Problem, is polynomial time reducible to the Integer Linear Program-
ming Problem.
52
 Formally, an instance  of the Satisability Problem is given by a Boolean
expression in conjunctive normal form (CNF):
 = C1 ^ C2 ^ ::: ^ Cm (3)
where each Ci (called a clause) is an or of Boolean literals. The question is
whether there is a Boolean assignment to the Boolean variables x1, x2 , : : :,
xn in  that makes the expression true.
We show how the input instance (3) of Satisability Problem is translated
into an input instance for Integer Linear Programming Problem.
Suppose that the clause Ci is
Ci = (xi1 _


_ xis _ xj1 _


_ xjt )
We then construct a linear constraint
xi1 +


+ xis + (1 xj1 ) +


+ (1 xjt )  1
Moreover, for each Boolean variable xj in , we have the constraints
xj  0 and xj  1
Here we let xj = 1 simulate the assignment xj = true and let xj = 0 simulate
the assignment xj = false. Therefore, the clause Ci is true if and only if
the corresponding linear constraint is satised for a 0-1 assignment to the
variables x1, x2 ,


, xn .
The objective function of the Integer Linear Programming Problem is
irrelevant in this reduction and can be dened arbitrarily. For example, we
can dene the objective function as
minfx1 + x2 +


+ xn g
which corresponds to nding a truth assignment for the Boolean expression
 such that the assignment has a minimum weight (i.e., the number of 1's
in the assignment is minimized).
It is easy to see that for the given input instance  for Satisability Prob-
lem, the corresponding input instance  () for Integer Linear Programming
Problem can be constructed in polynomial time. Moreover, if an optimal
solution is found for  (), then the Boolean expression  is certainly satis-
able thus the answer to  is YES. On the other hand, if no feasible solution
can be constructed for  () then  has no truth assignment so the answer
to  should be NO.
53
 Therefore, Satisability Problem is polynomial time reducible to Integer
Linear Programming Problem. Consequently, Integer Linear Programming
Problem is NP-hard.
As we have described in the previous Lecture Notes, the general Lin-
ear Programming Problem can be solved in polynomial time. The above
discussion shows that Integer Linear Programming Problem is much harder
than the general Linear Programming Problem. Our latter study will show
that Integer Linear Programming Problem is actually one of the hardest
optimization problems.

54
 CPSC-669 Computational Optimization
Lecture #13, September 27, 1995
Lecturer: Professor Jianer Chen
Scribe: Xiaotao Chen
Revision: Jianer Chen
13 NP-hard optimization problems
Recall the decision problem Partition that is dened as follows.
Partition
Input: A set S = fx1 ; x2; : : :; xn g of n integers
Question: Is there
P a0 xsubsetP S 0  S such that
i2S i = j 2S S 0 xj ?
It is well-known that the problem Partition is NP-complete.
We now introduce several optimization problems that are at least as hard
as Partition problem.
We start with an optimization version for the problem Partition that
is given by SubsetSum = (I; S; f; opt), where
 I = fhx1; x2; : : :; xn; Bi j xi; B : integersg
 S (hx1; : : :; xn; Bi) = fS 0  fx1; : : :; xng j Pxi2S0 xi  Bg
 f (hx1; : : :; xn; Bi; S 0) = Pxi2S0 xi
 opt : max
Theorem 13.1 The SubsetSum problem is NP-hard.
proof. We show a polynomial time reduction f from the problem Par-
tition to the problem SubsetSum.
Given an input instance  = fx1; x2; : : :; xn g for the problem Partition,
f () =Phx1; x2; : : :; xn; Bi is an instance for the problem SubsetSum, where
B =( n xi )=2. Now it is obvious that if an optimal solution to the
i=1
instance f () of SubsetSum is a subset S 0 of fx1; x2; : : :; xng such that
X Xn
xi = 21 xi
xi 2S 0 i=1

55
then  = fx1 ; : : :; xn g is a YES-instance for Partition, otherwise  is a
NO-instance for the problem.
Another popular optimization problem is Knapsack problem that is
formally dened by Knapsack = (I; S; f; opt), where
 I = fhs1; : : :; sn; v1; : : :; vn; Bi j si ; vj ; B : integersg
 S (hs1; : : :; sn; v1; : : :; vn; Bi) = fS  f1; : : :; ng j Pi2S si  Bg
 f (hs1; : : :; sn; v1; : : :; vn; Bi; S ) = Pi2S vi
 opt : max
An \application" of Knapsack problem can be described as follows. A
thief robbing a store nds n items. The ith item is worth vi dollars and
weighs si pounds. The thief wants to take as valuable a load as possible,
but he can carry at most B pounds in his knapsack. Now the thief wants to
decide what items he should take. Fortunately, the problem is NP-hard, as
we prove in the following theorem.
Theorem 13.2 The Knapsack problem is NP-hard.
proof. We construct a polynomial time reduction f from the problem
SubsetSum to the problem Knapsack.
Given an input instance  = hx1; : : :; xn ; B i for the problem Subset-
Sum, f () = hx1; : : :; xn; x1; : : :; xn ; B i is an input instance for the problem
Knapsack. Clearly, an optimal solution to the instance f () is a subset
S of Pf1; : : :; ng that satises the condition Pi2S xi  B and maximizes the
sum i2S xi . Thus, an optimal solution to the instance f () of Knapsack
is also an optimal solution to the instance  of SubsetSum.
We say that a collection of c subsets hS1; : : :; Sc i of f1; : : :; ng is a c-
partition of f1; : : :; ng if S1 [


[ Sc = f1; : : :; ng and all subsets S1 , : : :, Sc
are pairwisely disjoint.
We consider the optimization problem c-Processor Scheduling, which
is formally dened by c-Schedule = (I; S; f; opt), where
 I = fht1; : : :; tni j ti 's are integersg
 S (ht1; : : :; tni) = fhS1; : : :; Sci j hS1; : : :; Sci a c-partition of f1; : : :; ngg

56
  f (ht1; : : :; tni; hS1; : : :; Sci) = maxifPk2S tk g
i

 opt : min
Intuitively, suppose we are given n jobs such that the ith job takes exe-
cution time ti , and we want to distribute these jobs to c identical processors
so that the parallel nish time (i.e., the time at which all processors nish
their work) is minimized.
Theorem 13.3 The c-Processor Scheduling problem is NP-hard, for
c  2.
proof. We give a polynomial time reduction f from Partition problem
to c-Processor Scheduling problem.
Let  = hx1; : : :; xn i be an input instance
P for Partition problem. With-
out loss of generality, we assume that ni=1 xi is an even number | otherwise
 is clearly a NO-instance for Partition P Problem. We dene f () to be
n
hx1; : : :; xn; B3; : : :; Bci, where Br = ( i=1 ti )=2 for all r = 3; : : :; c. Clearly,
f () is an input instance for the c-Processor Scheduling problem. Now
P to verify that if an optimal solution to f () gives a parallel nish
it is easy
time ( ni=1 ti )=2, then hx1; : : :; xn i is a YES-instance for Partition prob-
lem, otherwise, hx1; : : :; xn i is a NO-instance for the problem.
Thus, all these three optimization problems described above are NP-
hard. By our believing that P 6= NP, they cannot be solved in polynomial
time. However, this does not obviate the need for solving these problems
because of their obvious applications. One possible approach is that we could
relax the requirement that we always nd the optimal solution. In practice,
a near-optimal solution will work ne in many cases. Of course, we expect
that the algorithms for nding the near-optimal solutions are ecient.
Denition 13.1 An algorithm A is an approximation algorithm for an op-
timization problem Q = (IQ; SQ; fQ ; optQ), if on any input instance x 2 IQ ,
the algorithm A produces a solution y 2 SQ (x).
Note that here we have put no requirement on the approximation quality
for an approximation algorithm. Thus, an algorithm that always produces
a \trivial" solution (for example, it simply returns the rst item for the
Knapsack problem) is an approximation algorithm. To measure the quality
of an approximation algorithm, we introduce the following concept.

57
Denition 13.2 An approximation algorithm A for an optimization prob-
lem Q = (IQ ; SQ; fQ; optQ) has an approximation ratio r(n), if on any input
instance x 2 IQ , the solution y produced by the algorithm A satises
Opt(x)  r(jxj) if opt = max
f (x; y ) Q
f (x; y )  r(jxj) if opt = min
Opt(x) Q

where Opt(x) is dened to be maxff (x; y ) j y 2 SQ (x)g if optQ = max and

to be minff (x; y ) j y 2 SQ (x)g if optQ = min.
Remark 13.3 By the denition, an approximation ratio is at least as large
as 1. It is easy to se that the closer the approximation ratio to 1, the better
the approximation quality of the approximation algorithm.

58
 CPSC-669 Computational Optimization
Lecture #14, September 29, 1995
Lecturer: Professor Jianer Chen
Scribe: Xiaotao Chen
Revision: Jianer Chen
14 The Knapsack problem
We start with an approximation algorithm for the Knapsack problem. Re-
call that the Knapsack problem is dened as
Knapsack
Input: hs1 ; : : :; sn ; v1; : : :; vn ; B i where all si ; vj ; B are integers
Output: A subset P S of f1; : : :; ng, such that Pi2S si  B
and i2S vi is maximized
We rst present an algorithm that solves the Knapsack problem pre-
cisely.
P s Tothesimplify the description, a subset S of f1; : : :; ng, we will call
P v theforvalue
i2S i size of S and i2S i of S . Let V = v1 + v2 +


+ vn .
Thus, there is no subset of f1; : : :; ng that can have value larger than V .
The algorithm goes as follows. For each index i and for each value j  V ,
we try to answer the question
Question K (i; j )
Is there a subset S of f1; : : :; ig such that the size of S is not
larger than B and the value of S is equal to j ?
The answer to Question K (i; j ) is \yes" if and only if at least one of the
following two cases is true: (1) there is a subset S 0 of f1; : : :; i 1g such that
the size of S 0 is not larger than B and the value of S is equal to j (in this
case, simply let S be S 0), and (2) there is a subset S 00 of f1; : : :; i 1g such
that the size of S 00 is not larger than B si and the value of S 00 is equal to
j vi (in this case, let S = S 00 [ fig). Therefore, if we are able to answer
Question K (i 1; j ) for all j , 0  j  V , we can answer Question K (i; j )
easily.
For small values i, the Question K (i; j ) seems easy. In particular, the
answer to K (0; j ) is always \no" for j > 0 and the answer to K (0; 0) is
\yes".
59
 The above discussion motivates the following dynamic programming al-
gorithm for solving the Knapsack problem. We rst compute K (0; j ) for all
j , then, inductively, compute each K (i; j ) based on the answer to K (i 1; j 0)
for all j 0. For each item K (i; j ), we associate it with a subset S in f1; : : :; ig
such that the size of S is not larger than B and the value of S is equal to j .
Now a potential problem arises. How do we handle two dierent wit-
nesses for a \yes" answer to the Question K (i; j )? More specically, suppose
that we nd two subsets S1 and S2 of f1; : : :; ig such that both of S1 and
S2 have size bounded by B and value equal to j , should we keep both of
them with K (i; j ), or ignore one of them? Keeping both can make K (i; j )
exponentially grow as i increases, which will signicantly slow down our al-
gorithm. Thus, we intend to ignore one of S1 and S2 . Which one do we want
to ignore? Intuitively, the one with larger size should be ignored (recall that
S1 and S2 have the same value). However, would ignoring the set cause a -
nal loss of the optimal solution? Fortunately, the following theorem ensures
that optimal solutions cannot get lost when we ignore the set with larger
size.
Theorem 14.1 Let S1 and S2 be two subsets of f1; : : :; ig such that S1 and
S2 have the same value, and the size of S1 is at least as large as the size
of S2. If S1 leads to an optimal solution S = S1 [ S3 for the Knapsack
problem, where S3  fi + 1; : : :; ng, then S 0 = S2 [ S3 is also an optimal
solution for the Knapsack problem.
proof. Let size(S ) and value(S ) denote the size and value of a subset S
of f1; : : :; ng, respectively. We have
size(S 0) = size(S2 ) + size(S3 ) and size(S ) = size(S1) + size(S3)
By the assumption that size(S1)  size(S2), we have size(S )  size(S 0).
Since S is an optimal solution, we have size(S 0)  B . Thus S 0 is also a
solution to the Knapsack problem. Moreover,
value(S 0) = value(S2) + value(S3) = value(S1) + value(S3) = value(S )
Thus, S 0 is also an optimal solution.
By Theorem 14.1, for two subsets S1 and S2 of f1; : : :; ig that both
witness the \yes" answer to Question K (i; j ), if the one of larger size leads
to an optimal solution, then the one with smaller size also leads to an optimal
60
solution. Therefore, ignoring the set of larger size will not lead to loss of
all optimal solutions. More specically, if we can derive an optimal solution
based on the set of larger size, then we can also derive an optimal solution
based on the set of smaller size using exactly the same procedure.
Now we are ready for the algorithm.
Algorithm 14.1 Knapsack-Dyn
Input: s1 ; : : :; sn ; v1; : : :; vn; B, all integers P
A subset S  f1; :::; ng, such that i2S si  B
Output:
P
and i2S vi is maximized
1. for i = 0 to n do
for j = 0 to V do
K [i; j ] = ;
2. K [0; 0] = ; f is the empty setg
3. for i = 0 to n 1 do
for j = 0 to V do
if K [i; j ] =6  then
Put(K [i; j ]; K [i + 1; j ]);
if size(K [i; j ]) + si+1  B then
v = j + vi+1; fj is the value of K [i; j ]g
Put(K [i; j ] [ fi + 1g; K [i + 1; v ]);
4. j =V;
while K [n; j ] =  do
j = j 1;
5. return K [n; j ].

Step 4 of the algorithm Knapsack-Dyn searches the last row from the
last column to nd the rst K [n; j ] that is not . Obviously, the value j
is the largest value a subset S of f1; : : :; ng can make under the restriction
that S has size bounded by B .
The subroutine Put(S0; K [i; j ]) is used to solve the multiple witness prob-
lem, where S0 is a subset of f1; : : :; ig such that S0 has value j . Details of
this subroutine is given as follows.
Algorithm 14.2 Put(S0; K [i; j ])
1. if K [i; j ] =  then
K [i; j ] = S0;
else if size(S0) < size(K [i; j ])
then K [i; j ] = S0.
61
 According to our discussion, it should be clear that the Algorithm 14.1
solves the Knapsack problem.
Theorem 14.2 The algorithm Knapsack-Dyn runs in time O(nV ).
proof. We show data structures on which the if statement in Step 3
can be executed in constant time. The theorem follows directly from this
discussion.
For each item K [i; j ], which is for a subset Sij of f1; : : :; ig, we associate
three parameters: (1) the size of Sij , (2) a marker mij indicating whether
i is contained in Sij , and (3) a pointer pij to an item K [i 1; j 0] in the
previous row such that the set Sij is derived from the set K [i 1; j 0]. Note
that the actual set Sij is not stored in K [i; j ].
With these parameters, the size of the set Sij can be directly read from
K [i; j ] in constant time. Moreover, it is also easy to verify that the subrou-
tine calls Put(K [i; j ]; K [i + 1; j ]) and Put(K [i; j ] [ fi + 1g; K [i + 1; v ]) can
also be performed in constant time by updating the parameters in K [i +1; j ]
and K [i + 1; v ].
This shows that steps 1-4 of the algorithm Knapsack-Dyn take time
O(nV ).
We must show how the actual optimal solution K [n; j ] is returned in
step 5. After we have decided the item K [n; j ] in step 5, which corresponds
to an optimal solution Snj that is a subset of f1; : : :; ng, we rst check the
marker mnj to see if Snj contains n, then follow the point pnj to an item
K [n 1; j 0], where we can check whether the set Snj contains n 1 and a
pointer to an item in the (n 2)nd row, and so on. In time O(n), we will
be able to \collect" all elements in Snj and return the actual set Snj .
It seems that we have developed a polynomial time algorithm that solves
the NP-hard optimization problem Knapsack. This is, in fact, not true
since the value V can be much larger than any polynomial of n.
Remark 14.1 We point out that the problem SubsetSum can be solved
by an algorithm very similar to Knapsack-Dyn, with running time O(nB )
on input instance fx1; : : :; xn ; B g. We leave the detailed implementation for
this algorithm as an exercise to the reader.

62
 CPSC-669 Computational Optimization
Lecture #15, October 2, 1995
Lecturer: Professor Jianer Chen
Scribe: Balarama Varanasi
Revision: Jianer Chen
15 Approximating Knapsack
We re-visit Knapsack problem and attempt to develop an approximation
algorithm that provides a solution of acceptable quality. Recall that the
Knapsack problem is dened as
Knapsack
Input: hs1 ; : : :; sn ; v1; : : :; vn ; B i where all si ; vj ; B are integers
Output: A subset P S of f1; : : :; ng, such that Pi2S si  B
and i2S vi is maximized
In the last lecture, we presented an algorithm Knapsack-Dyn that, on
an input instance X = hs1 ; : : :; sn ; v1; : : :; vn; B i of the KnapsackPproblem,
constructs an optimal solution for X in time O(nV ), where V = ni=1 vi . If
V is not bounded by any polynomial function of n, then the running time
of the algorithm is not polynomial. Is there a way to lower the value of V ?
Well, an obvious way is to divide each value vi by a suciently large number
K so that V is replaced by a smaller value V 0 = V=K . In order to let the
algorithm Knapsack-Dyn to run in polynomial time, we must have V 0  cnd
for some constants c and d, or equivalently, K  V=(cnd). Another problem
is that the value vi =K may not be an integer while by our denition, all
input values in an instance of Knapsack problem are integers. Thus, we
will take vi0 = bvi =K c. This gives a new instance X 0 for Knapsack problem
X 0 = hs1 ; : : :; sn; v10 ; : : :; vn0 ; Bi
where vi0 = bvi =K c, for i = 1; : : :; n. For K  V=(cnd) for some constants
c and d, the algorithm Knapsack-Dyn nds an optimal solution for X 0 in
polynomial time. Note that a solution to X 0 is also a solution to X and we
intend to \approximate" the optimal solution to X by an optimal solution
to X 0. Since the application of the oor function b
c, we lose precision thus

63
an optimal solution for X 0 may not be an optimal solution for X . How
much precision have we lost? Intuitively, the larger the value K , the more
precision we would lose. Thus, we want K to be as small as possible. On
the other hand, we want K to be as large as possible so that the running
time of the algorithm Knapsack-Dyn can be bounded by a polynomial. Now
a natural question is whether there is a value K that makes the algorithm
Knapsack-Dyn run in polynomial time and cause not much precision loss
so that the optimal solution to the instance X 0 is \close" to the optimal
solution to the instance X . For this, we need the following formal analysis.
Let S  f1; : : :; ng be an optimal solution to the instance X , and let
S  f1; : : :; ng be the optimal solution to the instance X 0 produced by the
0
algorithm Knapsack-Dyn. Note that S is also a solution to the P instance X 0
and that S 0Pis also a solution to the instance X . Let Opt(X ) = i2S vi and
Apx(X ) = j2S 0 vi be the objective function values of the solutions S and
S 0, respectively. Therefore, Opt(X )=Apx(X ) is the approximation ratio for
the algorithm we proposed. In order to bound the approximation ratio by
a given constant , we consider
X
Opt(X ) = vi
i2S
X vi
= K
i2S K
X vi
 K (b c + 1)
i2S K
X vi
 Kn + K b K c
i2S
X 0
= Kn + K vi
i2S
The last inequality is because the cardinality of the set S is bounded by n.
Now since S 0 is an optimal solution to X 0 = hs1 ; : : :; sn ; v10 ; : : :; vn0 ; B i,
we must have X X
vi0  vi0
i2S i2S 0
Thus,
X
Opt(X )  Kn + K vi0
i2S 0
X vi
= Kn + K b K c
i2S 0

64
 X vi
 Kn + K
i2S 0 K
= Kn + Apx(X ) (4)
This gives us the approximation ratio.
Opt(X ) Kn
Apx(X )  1 + Apx(X )
Without loss of generality, we can assume that si  B for all i = 1; : : :; n
(otherwise, the index i can be simply deleted from the input instance since
it can never make contribution to a feasible solution to X ). Thus, Opt(X )
is at least as large as max1in fvi g  V=n. From inequality (4), we have
Apx(X )  Opt(X ) Kn  Vn Kn
It follows that
Opt(X )  1 + Kn
Apx(X ) V Kn
n
2
= 1 + V KnKn2
Thus, in order to bound the approximation ratio by 1 + , it should be such
that 2
Kn  
V Kn2
This leads to K  (V )=(n2(1 + )).
Recall that to make the algorithm Knapsack-Dyn run in polynomial time
on the input instance X 0, we must have K  V=(cnd ) for some constants c
and d. Combining these two relations, we get c = 1 + 1=, and d = 2, and
the value
K = V=(cnd) = (1 + V1=)n2
makes the algorithm Knapsack-Dyn run in time O(n3(1+1=)) and produces
a solution S 0 to the instance X with approximation ratio bounded by .
We summarize the above discussion in the following algorithm and the-
orems.

65
Algorithm 15.1 Knapsack-Approx
Input: hs1; : : :; sn; v1; : : :; vn; Bi, and a constant
P .
0
A subset S  f1; :::; ng, such that
Output: i2S 0 si  B
1. Let K = (1+1V=)n2 ;
2. for i = 1 to n do vi0 = bvi=K c;
3. Apply algorithm Knapsack-Dyn on hs1 ; : : :; sn ; v10 ; : : :; vn0 ; B i
and find a subset S 0  f1; :::; ng;
4. Output S 0;

Theorem 15.1 For any input instance of the Knapsack problem, the algo-
rithm Knapsack-Approx runs in time O(n3 (1+1=)) and produces a solution
with approximation ratio bounded by 1 + .
Theorem 15.2 For any xed constant , there is an algorithm of running
time O(n3 ) that, on an input instance of the Knapsack problem, produces
a solution with approximation ratio bounded by 1 + .
This lecture concludes with the above result.

66
 CPSC-669 Computational Optimization
Lecture #16, October 4, 1995
Lecturer: Professor Jianer Chen
Scribe: Balarama Varanasi
Revision: Jianer Chen
16 Approximating Processor Scheduling
We continue the discussion of approximation algorithms.
We presented an O(n3 =) time approximation algorithm with approxi-
mation ratio  for any  > 0 for the Knapsack problem. Note that the time
complexity of this algorithm is polynomial in both the input size n and the
value 1=, which seems the best we can expect for an approximation algo-
rithm for an NP-hard optimization problem. This motivates the following
denition.
Denition 16.1 An optimization problem Q has a fully polynomial time
approximation scheme (FPTAS) if it has an approximation algorithm A
such that given hx; i, where x is an input instance of Q and  is a positive
constant, A nds a solution for x with approximation ratio bounded by 1+ 
in time polynomial in both n and 1=.
By the denition, the Knapsack problem has a fully polynomial time
approximation scheme. In this lecture, we illustrate the techniques for devel-
oping fully polynomial time approximation schemes for optimization prob-
lems by studying another important optimization problem, the c-Processor
Scheduling problem.
The approach for developing a fully polynomial time approximation
scheme for the c-Processor Scheduling problem is very similar to that
for the Knapsack problem: we rst develop a precise algorithm for the
problem such that the algorithm runs in time polynomial in both n and T ,
where T is a large number obtained from the input. Then we try to scale
T by dividing all numbers in the input by a large number K . By properly
choosing the value K , we can make the precise algorithm to run in polyno-
mial time and keep the approximation ratio bounded by a given constant
. Because of the similarity, some details in the algorithms and in the anal-
ysis are omitted. The reader is advised to refer to corresponding parts in

67
the study of the Knapsack problem and complete the omitted parts for a
better understanding.
Recall that the optimization problem c-Processor Scheduling is de-
ned by c-Schedule = (I; S; f; opt), where
 I = fht1; : : :; tni j ti 's are integersg
 S (ht1; :::; tni) = fhS1; :::; ScijhS1; :::; Sci is a c-partition of f1; :::; ngg
 f (ht1; : : :; tni; hS1; : : :; Sci) = max1ic fPk2Si tk g
 opt : min
P
Let T = ni=1 ti . Note that every scheduling (S1 ; : : :; Sc) of the n jobs
ht1; : : :; tni, where Sd is the subset of f1; : : :; ng that corresponds to the
jobs assigned to the dth processor, can be written as P a c-tuple (T1; : : :; Tc)
with 0  Td  T for all 1  d  c, where Td = h2Sd th is the total
execution time assigned to the dth processor. The c-tuple (T1; : : :; Tc) will
be called the time list for the scheduling (S1 ; : : :; Sc ). Moreover, each c-tuple
(T1; : : :; Tc) with 0  Td  T for all d = 1; : : :; c can be uniquely written
as a non-negative integer j less than or equal to (T + 1)c by the following
formula
j = T1(T + 1)c 1 + T2(T + 1)c 2 +


+ Tc 1(T + 1) + Tc (5)
Conversely, each non-negative integer j less than or equal to (T + 1)c can
be uniquely decomposed into a c-tuple (T1; : : :; Tc) with 0  Td  T for all
d = 1; : : :; c, using the formula (5).
Now as for the Knapsack problem, for each i, 0  i  n, and for each
non-negative integer j , where we suppose that the integer j is decomposed
into a c-tuple (T1; : : :; Tc) by the formula (5), we ask the question
Is there a scheduling of the rst i jobs ft1 ; : : :; tig that gives the
time list j = (T1; : : :; Tc)?
Note that for two dierent schedulings of the i jobs ft1 ; : : :; ti g that have
the same time list (T1; : : :; Tc), we can pick either of them without loss of
correctness.
Now we are ready to present the algorithm.
Algorithm 16.1 c-Scheduling-Dyn
Input: n jobs with execution time t1; : : :; tn, all integers

68
 Output: A scheduling of the jobs onn c
processors such
that the parallel finish time is minimized

f H [0::n; 0::(T + 1)c]

is a table such that the element H [i; j ]
is a scheduling ( 1 S ; : : :S
c ) of the first jobs 1 , i t : : :, ti
whose time list is = ( 1 j T ; : : :; T
c). g
1. T = Pni=1 ti ;
2. for i = 0 to n do
for j = 0 to (T + 1)c do
H [i; j ] = ;
3. H [0; 0] = (; : : :; ); f is the empty setg
4. for i = 0 to n 1 do
for j = 0 to (T + 1)c do
if H [i; j ] =6  then
Let H [i; j ] = (S1; : : :; Sc) is a scheduling of t1 , : : :,
ti, and j = (T1; : : :; Tc) is the time list for H [i; j ];
for d = 1 to c do
H [i + 1; jd] = (S1; : : :; Sd 1; Sd [ fi + 1g; Sd+1; : : :; Sc);
where jd = (T1; : : :; Td 1; Td + ti+1 ; Td+1; : : :; Tc);
5. Scan the nth row of the table H to find the scheduling
6  such that j = (T1; : : :; Tc) has the minimum
H [n; j ] =
parallel time;
6. Return H n; j
[ ].
We analyze the algorithm. As we did for the algorithm Knapsack-Dyn,
instead of storing the entire c-tuple (S1; : : :; Sc ) in H [i; j ], we simply keep a
marker that indicates which processor is assigned the ith job and a pointer
to the element H [i 1; j 0] such that the scheduling H [i; j ] of t1 , : : :, ti is
obtained from the scheduling H [i 1; j 0] of t1 , : : :, ti 1 by assigning the
job ti to a proper processor. By these data structures, each assignment
to the elements of the table H can be done in constant time. Moreover,
for each non-negative integer j less than or equal to (T + 1)c, by formula
(5), j can be uniquely decomposed using a constant number of division
and modulo operations into a c-tuple (T1; : : :; Tc) with 0  Td  T for all
d = 1; : : :; c (recall that c is a constant). Similarly, each c-tuple (T1; : : :; Tc)
with 0  Td  T for all d = 1; : : :; c can be converted in constant time into a
unique non-negative integer j less than or equal to (T + 1)c. In conclusion,
each execution of the if statement in the loop in step 4 takes constant time.
Consequently, the algorithm c-Scheduling-Dyn takes time O(nT c ).

69
 A fully polynomial time approximation scheme now is derived for the c-
Processor Scheduling problem based on algorithm c-Scheduling-Dyn.
The idea is the same as for the Knapsack problem: we rst scale the input
numbers to make T smaller then apply algorithm c-Scheduling-Dyn to the
scaled input.
Algorithm 16.2 c-Scheduling-Apx
Input: ht1; : : :; tn; i, t
all i 's are integers
Output: A scheduling of the jobs on n
processors c
1. Let K =  Pni=1 ti=(cn);
2. for i = 1 to n do t0i = dti =K e;
3. c
Apply algorithm -Scheduling-Dyn on input ht01; : : :; t0ni to
produce a scheduling ( 10 0
c) on 1 S ; : : :; S ht ; : : :; tni;
4. Output ( 10 0
S ; : : :; S
c );
Theorem 16.1 The algorithm c-Scheduling-Apx on input ht1; : : :; tn; i
produces a scheduling (S10 ; : : :; Sc0 ) with approximation ratio bounded by 1+ 
and runs in time O(nc+1 =c ).
proof. It is easy to see that the time complexity of the algorithm
c-Scheduling-Apx P is dominated P by step 3.
Since T0 = ni=1 t0i = O( ni=1 ti =K ) = O(n=), by our analysis, the algo-
rithm c-Scheduling-Dyn in step 3, thus the algorithm c-Scheduling-Apx,
runs in time O(nT0c ) = O(nc+1 =c ).
Now let (S1; : : :; Sc) be an optimal solution to the input instance X =
ht1; : : :; tni of the c-Processor Scheduling problem, and let (S10 ; : : :; Sc0 )
be the optimal solution to the input instance X 0 = ht01 ; : : :; t0n i obtained by
the algorithm c-Scheduling-Dyn. Note that (S1 ; : : :; Sc) is also a solution
to the instance ht01 ; : : :; t0n i and (S10 ; : : :; Sc0 ) is also a solution to the instance
ht1; : : :; tni.
For all d, 1  d  c, let
X X 0
Td = th Vd = th
h2Sd h2Sd
X X
Td0 = th Vd0 = t0h
h2Sd0 h2Sd0
Without loss of generality, suppose
T1 = 1max fT g
dc d
V2 = 1max fV g
dc d

70
 T30 = 1max fT 0 g
dc d
V40 = 1max fV 0g
dc d
Therefore, on instance ht1 ; : : :; tn i, the scheduling (S1; : : :; Sc) has parallel
nish time T1 and the scheduling (S10 ; : : :; Sc0 ) has parallel nish time T30 ;
and on instance ht01 ; : : :; t0n i, the scheduling (S1 ; : : :; Sc) has parallel nish
time V2 and the scheduling (S10 ; : : :; Sc0 ) has parallel nish time V40. The
approximation ratio given by the algorithm c-Processor-Apx is T30 =T1.
We have
X X X
T30 = th = K (th =K )  K t0h = KV30  KV40
h2S30 h2S30 h2S30
The last inequality is by the assumption V40 = max1dc fVd0 g.
Now since (S10 ; : : :; Sc0 ) is an optimal scheduling on instance ht01; : : :; t0n i,
we have V40  V2. Thus,
X X
T30  KV2 = K t0h = K dth =K e
h2S2 h2S2
X th
K ( + 1)  T2 + Kn  T1 + Kn
K
h2S2
The last inequality is by the assumption T1 = max1dc fTdg.
This gives us immediately
T30 =T1  1 + Kn=T1
P
It is easy to see that T1  ni=1 ti =c, and recall that K =  ni=1 ti =(cn),
P
we obtain Kn=T1  . That is, the scheduling (S10 ; : : :; Sc0 ) produced by the
algorithm c-Scheduling-Apx has approximation ratio bounded by 1+ .
Corollary 16.2 For a xed constant c, the c-Processor Scheduling
problem has a fully polynomial time approximation scheme.

71
 CPSC-669 Computational Optimization
Lecture #17, October 6, 1995
Lecture: Professor Jianer Chen
Scribe: Jennifer Walter
Revision: Jianer Chen
17 Which optimization problem has a FPTAS?
Let us rst review the denition of a fully polynomial-time approximation
scheme.
Denition 17.1 An optimization problem Q has a fully polynomial time
approximation scheme (FPTAS) if it has an approximation algorithm A
that on input hx; i, where x is an input instance of Q and  is a positive
number, gives a solution of approximation ratio bounded by 1 +  in time
polynomial in jxj and 1=.
This denition says that if a fully polynomial time approximation scheme
exists for an optimization problem Q, then there is a polynomial time ap-
proximation algorithm for the problem that can approximate the optimal so-
lution for the problem Q to any arbitrary precision. A fully polynomial time
approximation scheme seems the best solution we can hope to derive for an
NP-hard optimization problem. By our discussion in the previous lectures,
the optimization problems Subset Sum, Knapsack, and c-Processor
Scheduling have fully polynomial time approximation schemes.
Natural questions are how many problems we can devise a fully poly-
nomial time approximation scheme for, what are the kinds of possible fully
polynomial time approximation scheme solutions, and how it can be deter-
mined that a problem does not have a fully polynomial time approximation
scheme. To discuss these questions, we rst introduce a notation.
Denition 17.2 Let Q = hIQ; SQ; fQ; optQi be an optimization problem.
For each input instance x 2 IQ , dene OptQ (x) = optQ ffQ (x; y )jy 2 SQ (x)g.
That is, OptQ (x) is the value of the objective function fQ on input instance
x and an optimal solution to x.
We have the following very useful theorem.

72
Theorem 17.1 Let Q = hIQ; SQ; fQ; optQi be an optimization problem. If
there is a xed polynomial p such that for all input instances x 2 IQ , OptQ (x)
is bounded by p(jxj), then Q does not have a fully polynomial time approxi-
mation scheme unless Q can be precisely solved in polynomial-time.
proof. Let A be an approximation algorithm that is a fully polynomial
time approximation scheme for the optimization problem Q. We show that
Q can be precisely solved in polynomial time.
By the denition, we can suppose that the running time of A is nc =d ,
where c and d are xed constants. Moreover, by the condition given in the
theorem, we can assume that OptQ (x)  nh , where h is also a xed constant.
First assume that optQ = min. For an input instance x 2 IQ , let A(x) be
the objective function value on the input x and the solution to x produced
by the algorithm A. Thus, we know that for any  > 0, the algorithm A
produces in time nc =d a solution with approximation ratio A(x)=Opt(x) 
1 + . Also note that A(x)=Opt(x)  1.
Now, let  = 1=nh+1 , then the algorithm A produces a solution with
approximation ratio bounded by
A(x)  1 + 1
1  Opt (x) nh+1
which gives
Opt(x)  A(x)  Opt(x) + Opt(x)=nh+1
Since both Opt(x) and A(x) are integers, and Opt(x)  nh implies that
Opt(x)=nh+1 is a number strictly less than 1, we conclude that
Opt(x) = A(x)
That is, the algorithm A actually produces an optimal solution to the input
instance x. Moreover, the running time of A is bounded by nc =(1=nh+1 )d =
nc+hd+d , which is a polynomial of n.
The case that optQ = max can be proved similarly. Note that in this case,
we should also have A(x)  nh . Thus, in time nc =(1=nh+1 )d = nc+hd+d , the
algorithm A produces a solution to x with the value A(x) such that
1  Opt(x)=A(x)  1 + 1=nh+1
which gives
A(x)  Opt(x)  A(x) + A(x)=nh+1
Now since A(x)=nh+1 < 1, we conclude Opt(x) = A(x).
73
 In particular, this theorem says that if OptQ (x) is bounded by a poly-
nomial of the input length jxj and Q is known to be NP-hard, then Q does
not have a fully polynomial time approximation scheme unless P = NP.
Theorem 17.1 is actually very powerful. Most NP-hard optimization
problems satisfy the condition stated in the theorem, thus we can derive
directly that these problems have no fully polynomial time approximation
scheme. We will give a few examples below to illustrate the power of Theo-
rem 17.1.
Consider the following problem:
Independent Set IS = hI; S; f; opti
I : set of all graphs G = (V; E )
S (G): the collection of subsets S of vertices of G such that no
two vertices in S are adjacent
f (G; S ): the number of vertices in S
opt: max
Independent Set problem has many applications in networking design
and scheduling. A trivial solution to the Independent Set problem is to
pick one single vertex, or a small number of vertices from the graph which are
not adjacent. The problem is more dicult for a very large set of vertices.
In fact, this is well-known that the Independent Set problem is NP-hard.
It is easy to apply Theorem 17.1 to show that the Independent Set
problem has no fully polynomial time approximation scheme. In fact, the
value of the objective function is bounded by the number of vertices in the
input graph G, which is certainly bounded by a polynomial of the input
length jGj.
There are many other graph problems (actually, most graph problems)
like the Independent Set problem that ask to optimize a subset of ver-
tices or edges of the input graph. For all these problems, we can conclude
directly from Theorem 17.1 that they do not have a fully polynomial time
approximation scheme unless they can be solved precisely in polynomial
time.
Let us consider another example of a problem for which no fully polyno-
mial time approximation scheme exists.
Bounded-Time Processor Scheduling

74
 Input: ft1; t2; : : :; tn; Bg, all integers where each ti is the exe-
cution time for the ith job and B is a restriction on the parallel
nish time
Output: A scheduling of the n jobs on m processors such that
the parallel nish time is bounded by B and m is minimized
The Bounded-Time Processor Scheduling problem is commonly
called Bin Packing problem, which is known to be NP-hard. Given an
input instance for the Bounded-Time Processor Scheduling problem,
either we can conclude immediately that there is no such scheduling (if any
input job has execution time larger than B ), or we know the output value
m is bounded by n (i.e., in the worst case, each processor is assigned with a
single job). In any case, we have Opt(x) bounded by n. By Theorem 17.1,
we conclude directly that the Bounded-Time Processor Scheduling
problem has no fully polynomial time approximation scheme unless P =
NP.
Remark 17.3 Although this version of the scheduling problem has no fully
polynomial time approximation scheme, the majority of the problems for
which a fully polynomial time approximation scheme exists are scheduling
problems.
What if the condition of Theorem 17.1 does not hold? Can we still
derive a conclusion of nonexistence of a fully polynomial time approximation
scheme for an optimization problem? We study this problem starting with
the famous Traveling Salesman problem (TSP), and will derive general
rules for this kind of optimization problems.
Traveling Salesman (TSP)
Input: a weighted complete graph G
Output: a simple cycle through all vertices of G (such a simple
cycle is called a traveling salesman tour) and the weight of the
cycle is minimized
The Traveling Salesman problem obviously does not satisfy the con-
dition stated in Theorem 17.1. For example, if all edges of the input graph
G of n vertices have weight of order (2n ), then the weight of the min-
imum traveling salesman tour is
(n2n ) while a binary representation of
the input graph G has length bounded by O(n3 ) (note that the length of
75
the binary representation of a number of order (2n ) is O(n) and G has
O(n2) edges). Therefore, Theorem 17.1 does not apply to the Traveling
Salesman problem.
To show the non-approximability of the Traveling Salesman problem,
we rst consider a simpler version of the Traveling Salesman problem,
which is dened as follows.
Traveling Salesman 1-2 (TSP(1,2))
Input: a weighted complete graph G such that the weight of
each edge of G is either 1 or 2
Output: a traveling salesman tour of minimum weight
Theorem 17.2 The Traveling Salesman 1-2 problem is NP-hard.
proof. We present a polynomial time reduction that transforms the well-
known NP-complete problem Hamiltonian Circuit to the Traveling
Salesman 1-2 problem.
By the denition, for each undirected unweighted graph G of n vertices,
the Hamiltonian Circuit problem asks if G contains a Hamiltonian cir-
cuit, i.e., a simple cycle of length n.
Given an input instance G = (V; E ) for the Hamiltonian Circuit
problem, we add edges to G to make a weighted complete graph G0 =
(V; E [ E 0) such that for each edge e 2 E of G0 that is in the original graph
G, we assign a weight 1 and for each edge e0 2 E 0 of G0 that is not in
the original graph G, we assign a weight 2. The graph G0 is certainly an
input instance of the Traveling Salesman 1-2 problem. Now, let T be a
minimum weighted traveling salesman tour in G0. It is easy to verify that
the weight of T is equal to n if and only if the original graph G contains a
Hamiltonian circuit.
This completes the proof.
Theorem 17.1 can apply to the Traveling Salesman 1-2 problem
directly.
Theorem 17.3 The Traveling Salesman 1-2 problem has no fully poly-
nomial time approximation scheme unless P = NP.
proof. Since the weight of a traveling salesman tour for an input instance
G of the Traveling Salesman 1-2 problem is at most 2n, assuming that
76
G has n vertices, the condition stated in Theorem 17.1 is satised by the
Traveling Salesman 1-2 problem. Now the theorem follows from Theo-
rem 17.1 and Theorem 17.2.
Now we are ready for a conclusion on the approximability of the Trav-
eling Salesman problem in its general form.
Theorem 17.4 The Traveling Salesman problem has no fully polyno-
mial time approximation scheme unless P = NP.
proof. Since each input instance for the Traveling Salesman 1-2
problem is also an input instance for the Traveling Salesman problem, a
fully polynomial time approximation scheme for the Traveling Salesman
problem should also be a fully polynomial time approximation scheme for
the Traveling Salesman 1-2 problem. Now the theorem follows from
Theorem 17.3.

77
 CPSC-669 Computational Optimization
Lecture #18, October 9, 1995
Lecture: Professor Jianer Chen
Scribe: Jennifer Walter
Revision: Jianer Chen
18 Strong NP-hardness
We continue the discussion on what conditions will make an optimization
problem have a fully polynomial time approximation scheme. In the last
lecture, we have seen that if the optimal value OptQ (x) is always bounded
by a polynomial of the input length of x, then the problem Q has no fully
polynomial time approximation scheme unless Q can be solved precisely in
polynomial time. We have also studied the Traveling Salesman problem,
which does not satisfy the above condition, and developed a technique to
show that the Traveling Salesman problem has no fully polynomial time
approximation scheme. We started by a restricted version of the Trav-
eling Salesman problem, the Traveling Salesman 1-2 problem, and
showed that it satises the above condition and is also NP-hard. Thus,
the Traveling Salesman 1-2 problem has no fully polynomial time ap-
proximation scheme. From this we derived that the original Traveling
Salesman problem does not have a fully polynomial time approximation
scheme. In this lecture, we will formalize this technique and extend it to
other optimization problems.
For many optimization problems, such as those we have previously dis-
cussed as Subset Sum, Knapsack, c-Processor Scheduling, and Trav-
eling Salesman, an input instance is always associated with numbers. In-
deed, it is natural to dene a number in the problem statement for these
problems.
Denition 18.1 Suppose Q = hIQ; SQ; fQ; optQi is an optimization prob-
lem. For each input instance x 2 IQ we can dene:
 length(x) = the length of binary representation of x; and
 max(x) = the largest number that appears in input x.

78
 In particular, if no number appears in the input instance x, we dene
max(x) = 0.
Denition 18.1 can vary by some degree without loss of the generality
of our discussion. For example, length(x) can also denote the length of the
decimal representation in the input x or of any other xed base representa-
tion in the input x, and max(x) can be dened to be the sum of all numbers
appearing in the input x. Our discussion below will be valid for any of these
variations. The point is that for two dierent denition systems (length(x),
max(x)) and (length0 (x), max0 (x)), we require that length(x) and length0 (x)
are polynomially related and that max(x) and max0 (x) are polynomially
related for all input instances x.
Denition 18.2 An optimization problem Q = hIQ; SQ; fQ; optQi is a non-
number problem if for all x 2 IQ , max(x)  p(length(x)), where p is a xed
polynomial. If there is no such a polynomial p exists, then Q is called a
number problem.
According to the denition, Subset Sum, Knapsack, c-Processor
Scheduling, and Traveling Salesman problems are all number prob-
lems. Independent Set is a non-number problem.
Denition 18.3 Let Q = hIQ; SQ; fQ; optQi be an optimization problem
and let q be any function. Dene an optimization problem Qq to be the sub-
problem of Q such that Qq = hIQ0 ; SQ0 ; fQ0 ; opt0Qi, where IQ0  IQ , SQ0 = SQ ,
fQ0 = fQ and opt0Q = optQ, and for all x 2 IQ0 , max(x)  q (length(x)).
In other words, for all input instances x of Qq , max(x) is bounded by
q (length(x)).
The following denition was rst introduced and studied by Garey and
Johnson.
Denition 18.4 An optimization problem Q = hIQ; SQ; fQ; optQi is NP-
hard in the strong sense if Qq is NP-hard for some polynomial q .
The Traveling Salesman problem is an example of optimization prob-
lems that are NP-hard in the strong sense, as shown by the following theo-
rem.
Theorem 18.1 The Traveling Salesman problem is NP-hard in the strong
sense.

79
proof. If we denote by Q the Traveling Salesman problem, then Q2
corresponds to the Traveling Salesman 1-2 problem. By Theorem 17.2,
the Traveling Salesman 1-2 problem is NP-hard. Now by the above
denition, the Traveling Salesman problem is NP-hard in the strong
sense.
Remark 18.5 Every non-number NP-hard optimization problem Q is NP-
hard in the strong sense. This is because for every non-number NP-hard
optimization problem Q, Q = Qp for some polynomial function p. This
implies that Qp is NP-hard, which, by the denition, further implies that Q
is NP-hard in the strong sense.
Theorem 18.2 Subset-Sum, Knapsack, and c-Processor Schedul-
ing problems are not NP-hard in the strong sense unless P = NP.
proof. Let Q be any one of these problems. From previous lectures, we
know that there is an algorithm A such that for an input instance x of Q, the
algorithm A constructs an optimal solution to x in time O(nc V d ) for some
constants c and d, where V  n
max(x). Therefore,0 the algorithm A solves
the optimization problem Q in time O((length(x))c (max(x))d ), where c0 is
a constant.
If Q is NP-hard in the strong sense, then Qp is NP-hard for some
xed polynomial p. However, for all input instances x of Qp , max(x) 
p((length)(x)). Thus, the algorithm A constructs an optimal solution for
each input instance x of Qq in time
O((length(x))c0 (max(x))d) = O((length(x))c0 ((p(length(x)))d))
which is bounded by a polynomial of length(x). Thus, the problem Qp is
solvable in polynomial time, which implies P = NP.
The following theorem serves as a fundamental theorem for showing
which number problem has no fully polynomial time approximation scheme.
We say that a two-parameter function f (x; y ) is a polynomial of x and y if
f (x; y ) can be written as a nite sum of the terms of form xcy d, where c and
d are non-negative integers.
Theorem 18.3 Let Q = hIQ; SQ; fQ; optQi be an optimization problem that
is NP-hard in the strong sense. Suppose that for all x 2 IQ , OptQ (x) is

80
bounded by a polynomial of length(x) and max(x). Then Q has no fully
polynomial time approximation scheme unless P = NP.
proof. The proof of this theorem is very similar to the discussion we have
given for the Traveling Salesman problem in the last lecture.
Since Q is NP-hard in the strong sense, Qq is NP-hard for a polynomial
q. Let Qq = hIQ0 ; SQ; fQ; optQi such that for each input instance x 2 IQ0 , we
have max(x)  q (length(x)). Combining this condition with the condition
stated in the theorem that OptQ (x) is bounded by a polynomial of length(x)
and max(x), we derive that OptQ (x) is bounded by a polynomial of length(x)
for all input instances x 2 IQ0 . Now by Theorem 17.1, the problem Qq has
no fully polynomial time approximation scheme unless P = NP. Since each
input instance of Qq is also an input instance of Q, a fully polynomial time
approximation scheme for Q is also a fully polynomial time approximation
scheme for Qq . Now the theorem follows.
Remark 18.6 How common is the situation that OptQ(x) is bounded by a
polynomial of length(x), and max(x)? In fact, this situation is fairly com-
mon because for most optimization problems, the objective function value
is dened through additions or constant number of multiplications on the
numbers appearing in the input instance x, which is certainly bounded by
a polynomial of length(x) and max(x). Of course, the condition is not uni-
versely true for general optimization problems. For example, an objective
function can be simply dened to be the exponentiation of the sum of a sub-
set of input values, which cannot be bounded by any polynomial of length(x)
and max(x).
A general technique for showing the strong NP-hardness for an opti-
mization problem Q is to pick an NP-complete problem L and show that L
is polynomial time reducible to Qq for some polynomial q . Our polynomial
time reduction from the Hamiltonian Circuit problem to the Traveling
Salesman 1-2 problem given in the last lecture well illustrates this idea.
We give another example of optimization problems that are NP-hard in
the strong sense.
Multi-Processor Scheduling (MPS)
Input: ft1; t2 ; : : :; tn ; mg, all integers, where ti is the execution
time for the ith job
Output: a scheduling of the n jobs on m identical processors
such that the parallel nish time is minimized
81
 The Multi-Processor Scheduling problem is NP-hard in the strong
sense. In fact, the following restricted version of the Multi-Processor
Scheduling problem is NP-hard in the strong sense.
Three-Partition
Input: ft1 ; t2; : : :; t3m ; mg, all integers, where ti is the execution
time for the ith job
Output: a scheduling of the 3m jobs on m identical processors
such that the parallel nish time is minimized
The reader is advised to read Section 4.2.2 in Computers and Intractabil-
ity: A Guide to the Theory of NP-Completeness by M. Garey and D. John-
son, for a detailed proof that the Three-Partition is NP-hard in the
strong sense. Chapter 4 of the above book also contains excellent discussion
on strong NP-hardness of optimization problems.
We should point out that for the Multi-Processor Scheduling prob-
lem, when the number of processors is xed by a constant c, the problem
has a fully polynomial time approximation scheme, as we discussed in the
previous lectures on the c-Processor Scheduling problem. However, if
the number m of processors is given as a variable in the input, then the
problem becomes NP-hard in the strong sense. By Theorem 18.3, the prob-
lem has no fully polynomial time approximation scheme (it is easy to verify
that the condition that Opt(x) is bounded by a polynomial of length(x) and
max(x) is satised by this problem).

82
 CPSC-669 Computational Optimization
Lecture #19, October 11, 1995
Lecture: Professor Jianer Chen
Scribe: Jennifer Walter
Revision: Jianer Chen
19 Absolute approximability
We have seen a number of NP-hard optimization problems for which we can
derive a polynomial time approximation algorithm with approximation ratio
bounded by an arbitrary constant  > 0. In this section, we will discuss the
approximability of optimization problems in terms of a dierent measure |
the absolute dierence.
Denition 19.1 Let Q = hIQ; SQ; fQ; optQi be an optimization problem
and let d(n) be a function. We say that Q can be approximated with an
absolute dierence d(n) in polynomial time if there is a polynomial time
approximation algorithm A for Q such that for any input instance x of Q,
the algorithm A produces a solution y to x such that
jOpt(x) fQ(x; y)j  d(jxj)
We start the discussion with the famous planar graph coloring problem.
Planar Graph Coloring
Input: a planar graph G
Output: a coloring of the vertices of G such that no two
adjacent vertices are colored with the same color and the number
of colors used is minimized.
Theorem 19.1 The Planar Graph Coloring problem is NP-hard.
proof. In fact, the decision problem Planar Graph 3-Colorability:
\given a planar graph G, can G be colored with at most 3 colors?" is NP-
complete. It is straightforward that the Planar Graph 3-Colorability
problem is polynomial time reducible to the Planar Graph Coloring
problem.

83
Theorem 19.2 The Planar Graph Coloring problem can be approxi-
mated in polynomial time with an absolute dierence 2.
proof. First note that there is a well-known and simple process that
colors any planar graph with at most 5 colors. Moreover, the process can
be implemented by a polynomial time algorithm.
Therefore, given a planar graph G, we rst check if G is 2-colorable |
this is equivalent to checking if G is a bipartite graph and can be done in
linear time. If G is 2-colorable, then we color G with 2 colors and obtain an
optimal solution. Otherwise, we need at least 3 colors and we call the above
algorithm to color G with at most 5 colors.
Remark 19.2 By the famous Four-Color Theorem, every planar graph can
be colored with at most 4 colors. Therefore, the absolute dierence in The-
orem 19.2 can actually be replaced by 1. However, since the Four-Color
Theorem is too involved, we rather use a much simpler Five-Color Theorem
here.
Thus, the Planar Graph Coloring problem can be approximated
with a constant absolute dierence. On the other hand, the approximation
algorithm does not seem very good in term of the approximation ratio. For
example, for a planar graph that is 3-colorable, the algorithm can only guar-
antee a 4-coloring solution. Thus, the approximation ratio for this algorithm
is at least 4=3 > 1:3. The reason for this is that the optimal value of an
instance of the problem is always bounded by a constant. Thus, even a
small absolute dierence makes a signicant error in the approximation ra-
tio. Next we give another example for which optimal values are not bounded
while the problem still has very good approximation algorithm in terms of
the absolute dierence.
Graph Edge Coloring
Input: a graph G
Output: a coloring of the edges of G such that no two adjacent
edges are colored with the same color and the number of colors
used is minimized.
Remark 19.3 The Graph Edge Coloring problem is NP-hard.

84
 Given a graph G, let v be a vertex of G. Dene deg (v ) to be the degree
of the vertex and dene deg (G) to be the maximum deg (v ) over all vertices
v of G.
The following lemma follows directly from the denition.
Lemma 19.3 Every edge coloring of a graph G uses at least deg(G) colors.
Since deg (G) can be arbitrarily large, the optimal value for an instance
of the Graph Edge Coloring problem is not bounded by any constant.
This is the dierence to the case of the Planar Graph Coloring problem.
The next lemma may look more surprising.
Lemma 19.4 There is a polynomial time algorithm that colors a given
graph G with at most deg (G) + 1 colors.
proof. Let G be the input graph. To simplify the expression, let d =
deg (G). We present an algorithm that colors the edges of G using at most
d + 1 colors.
The algorithm has the following framework.
Algorithm 19.1 Edge-Coloring
Input: a graph G
Output: an edge coloring of G
1. let G G
0 = with all edges deleted;
f Suppose that the edges of are G e1, : : :, em g
2. for i = 1 to m do
Gi = Gi 1 [ feig;
color the edges of Gi using at most d+1 colors;

We need to explain how the graph Gi can be colored with at most d + 1

colors. Inductively, suppose that we have colored the edges of Gi 1 using
at most d + 1 colors. Now Gi = Gi 1 [ fei g, where suppose ei = (v1; w).
Thus, we have all edges of Gi except ei colored properly using at most d + 1
colors.
We say that a vertex u in Gi misses a color c if no edge incident on u is
colored with c. Since we have d + 1 colors and each vertex of Gi has degree
at most d, every vertex of Gi misses at least one color.
If both vertices v1 and w miss a common color c, then we simply color
ei = (v1; w) with c and we obtain a valid coloring for the graph Gi.

85
 (miss c4 )
(miss c3) vs4
vs3 vs5
ee c % %
c
(miss c2) v2 sHH c e2
3
%c4
H1H e %
HHeHes%%
v1 s w
(miss c1 )
h=5
Figure 2: A fan structure
So we suppose that there is no color that is missed by both v1 and w.
Let c1 be the color missed by v1 and c0 be the color missed by w, c1 6= c0 .
Since c1 is not missed by w, there is an edge (v2 ; w) colored with c1.
Now if v2 and w have a common missed color, we stop. If v2 and w have no
common missed color, then let c2 be a color missed by v2 | c2 is not missed
by w. Now let (v3 ; w) be the edge colored with c2 .
Inductively, suppose that we have constructed a "fan" that consists of h
neighbors v1 , : : :, vh of w and h 1 dierent colors c1, : : :, ch 1 , such that
(see Figure 2)
 for all j = 1; : : :; h 1, the vertex vj misses color cj and the edge
(vj +1 ; w) is colored with the color cj ;
 none of the vertices v1, : : :, vh 1 have a common missed color with w;
 for all j = 1; : : :; h 1, the vertex vj does not miss any of the colors
c1, : : :, cj 1 .
There are three possible cases.
Case 1. the vertex vh does not miss any of the colors c1, : : :, ch 1 and
vh has no common missed color w.
Then let ch be a color missed by vh . Since ch is not missed by w, there is
an edge (vh+1 ; w) colored with ch . Thus, we have expanded the fan structure
by one more edge.
Since the degree of the vertex w is nite, Case 1 must fail at some stage
and one of the following two cases should happen.
Case 2. the vertex vh has a common missed color c0 with w.

86
 vs4
vs3 vs5
e %
e c
v2 sHH c e3
c 4 %
H2HHee %%%c0
v1 s HHes%
c1 w
h=5
Figure 3: In case vh and w miss a common color c0
Then we change the coloring of the fan by coloring (vh ; w) with c0, and
coloring (vi ; w) with ci , for i = 1; : : :; h 1 (see Figure 3). It is easy to verify
that this gives a valid edge coloring for the graph Gi = Gi 1 [ fei g.
Case 3. the vertex vh misses a color cs, 1  s  h 1.
Let c0 be a color missed by w. We start from the vertex vs . Since vs has
no common missed color with w, there is an edge (vs ; u1) colored with c0.
Now if u1 does not miss cs , there is an edge (u1; u2) colored with cs , now we
look at vertex u2 and see if there is an edge colored with c0, and so on. By
this, we obtained a path Ps whose edges are alternatively colored by c0 and
cs. The path has the following properties: (1) the path Ps must be a simple
path since each vertex of the graph Gi 1 has at most two edges colored with
c0 and cs. Thus, the path Ps must be a nite path; (2) the path Ps cannot
be a cycle since the vertex vs misses the color cs ; and (3) the vertex w is
not an interior vertex of Ps since w misses the color c0 .
Let Ps = fvs ; u1; : : :; utg, where vs misses the color cs , ut misses one of
colors cs and c0, and uj , j = 1 : : :; t 1, misses neither cs nor c0.
If ut 6= w, then interchange the colors c0 and cs on the path Ps to make
the vertex vs miss c0. Then color (vs ; w) with c0 and color (vj ; w) with cj ,
for j = 1; : : :; s 1 (see Figure 4). It is easy to verify that this gives a valid
edge coloring for the graph Gi = Gi 1 [ fei g.
If ut = w, we must have ut 1 = vs+1 . Then we grow a c0 -cs alternating
path Ph starting from the vertex vh , which also misses the color cs . Again
Ph is a nite simple path. Moreover, the path Ph cannot end at the vertex
w since no vertex in Gi 1 is incident on more than two edges colored with
c0 and cs and the vertex w misses the color c0. Therefore, similar to what
we did for vertex vs , we interchange the colors c0 and cs on the path Ph to
make vh miss c0 . Then color (vh ; w) with c0 and color (vj ; w) with cj for

87
 s (misss c3) s s
c0 (miss c4) c3
c3 (miss c3 ) vs4 (miss c3 ) c0 vs4
vs3 vs5 vs3 vs5
s
eec0 %
s
c3ee %
c c3 % c c 3 %
v sH e2
(miss2c2 ) HHc1H e
%c4 =) v2 sHH c e0 %c4
HHeHes%% % H2H e
HHeHes%% %
v1 s w v1 s c w
(miss c1 ) 1
h = 5 and s = 3
Figure 4: Extending a c0-cs alternating path Ps from vs not ending at w
(miss sc0) s s s
s
c3 @
s c3 s
c3 s@ c0
c0 c0 @sv c0 c0 c0 @vs4 c3
4
(miss c4) s v3 se
(miss c3 ) v3e %v5 %
s s
v5
e c2 c3 %(miss c3 )
HHHce2 ec3e c4 %%c0 %
2 HH c e
v
%%c4 v2
s s
(miss c ) 2 H1HHee % =) H
v1 s
HHes% v1 s c1 HHeH es%%
(miss c1 ) w w
h = 5 and s = 3
Figure 5: Extending a c0-cs alternating path Ph from vh not ending at w
j = 1; : : :; h 1 (see Figure 5). It is easy to verify that this gives a valid
edge coloring for the graph Gi = Gi 1 [ fei g.
Therefore, starting with an edge coloring of the graph Gi 1 using at
most d + 1 colors, we can always derive a valid edge coloring for the graph
Gi = Gi 1 [ fei g using at most d + 1 colors. It is also easy to see that this
process can be implemented by a polynomial time algorithm. We leave the
detailed implementation of this process to the interested reader.
Now we conclude that the algorithm Edge-Coloring runs in polynomial
time and produces a valid edge coloring using at most d + 1 colors for the
graph G.
Theorem 19.5 The Graph Edge Coloring problem can be approxi-
mated within an absolute dierence of 1 in polynomial-time.

88
proof. Follows directly from Lemma 19.3 and 19.4.
Remark 19.4 Theorem 19.5 seems to give the best possible polynomial
time approximation algorithm for the NP-hard Graph Edge Coloring
problem. On the other hand, this algorithm does not provide a fully poly-
nomial time approximation scheme for the problem. Indeed, the decision
problem Graph Edge 3-Colorability \given a graph G, can the edges
of G be colored using no more than 3 colors" is NP-complete. Thus, the algo-
rithm from Theorem 19.5 can only guarantee a 4-coloring for an instance of
the Graph Edge 3-Colorability problem, which has an approximation
ratio at least 4=3 > 1:3.
It is natural to ask whether the optimization problems that have fully
polynomial time approximation scheme should have good approximation
algorithms in terms of absolute dierence. It is, in fact, not very dicult to
show that this is not always the case.
Recall the Knapsack problem.
Knapsack
Input: (s1 ; : : :; sn ; v1; : : :; vn; B ), all integers
Output : a subset S of f 1 ; : : :; ng such that
P s  B and
P v is maximized i2S i
i2S i
Theorem 19.6 There is no polynomial time approximation algorithm for
the Knapsack problem that guarantees an absolute dierence 2n unless P
= NP.
proof. Suppose that A is a polynomial time approximation algorithm for
the Knapsack problem Q = hI; S; f; opti such that for any input instance
X of Q, A produces a solution S such that jOpt(X ) A(X )j  2n , where
A(X ) = f (X; S ). We show how we can use this algorithm to solve the
Knapsack problem in polynomial time.
Given an input instance X = (s1; : : :; sn ; v1; : : :; vn ; B ) for Q, we con-
struct X 0 = (s1 ; : : :; sn ; v12n+1 ; : : :; vn 2n+1 ; B ) (i.e. scale the values vi to be
a multiple of 2n+1 so that a dierence of 2n between two values makes no
dierence).
Now apply the algorithm A to X 0 to get a solution S with value A(X 0) =
f (X 0; S ). According to our assumption, jOpt(X 0) A(X 0)j  2n . Since
89
both Opt(X 0) and A(X 0) are multiples of 2n+1 , we conclude that Opt(X 0) =
A(X 0), that is, the solution S is an optimal solution to the instance X 0.
Moreover, it is easy to see that S is also a solution to the instance X and
Opt(X 0) = 2n+1 Opt(X ) and A(X 0) = 2n+1A(X ). Therefore, S is also an
optimal solution for the instance X .
By our assumption, the algorithm A runs in polynomial time. It is
also easy to see that we can construct the instance X 0 from the instance
X in polynomial time. Therefore, the above process constructs an optimal
solution for X in polynomial time. Consequently, the Knapsack problem
can be solved in polynomial time. Since the Knapsack problem is NP-hard,
it follows that P = NP.
This proof for Theorem 19.6 can be easily extended to other number
problems such as the c-Processor Scheduling problem and the Subset
Sum problems.
The main reason that Theorem 19.6 holds for many number problems is
that we can scale the numbers in the input instances so that a small abso-
lute dierence would make no dierence for the scaled instance. However,
what about non-number problems? In particular, is there a similar theorem
for optimization problems whose instances contain no number at all? We
demonstrate a technique for this via the study of an optimization problem
related to graph embeddings.
Graph embeddings can be studied using graph rotation systems. A rota-
tion at a vertex v is a cyclic permutation of the edge-ends incident on v . A
list of rotations, one for each vertex of the graph, is called a rotation system.
An embedding of a graph G in an orientable surface induces a rotation
system, as follows: the rotation at vertex v is the cyclic permutation corre-
sponding to the order in which the edge-ends are traversed in an orientation-
preserving tour around v . Conversely, it is known that every rotation system
induces a unique embedding of G into an orientable surface. In the follow-
ing, we will interchangeably use the phrases \an embedding of a graph" and
\a rotation system of a graph".
The genus ((G)) of the rotation system (G) is dened by the Euler
polyhedral equation
jV j jE j + jF j = 2 2 ((G))
where jF j is the number of faces in the embedding (G). It can be proved
that the value ((G)) is actually the number of \holes" of the surface on
which the embedding (G) is realized. Consequently, ((G)) is always a
90
non-negative integer. There is a linear time algorithm that, given a rotation
system (G) for a graph G, traces the boundary walks of all faces in the ro-
tation system. Therefore, given a rotation system (G), the genus ((G))
of (G) can be computed in linear time.
Now we are ready to state the following problem.
Graph Genus
Input: a graph G
Output: an embedding (G) of G such that the genus ((G))
is minimized. Such a value is called the minimum genus of the
graph G, written as min(G)
It is known that the Graph Genus problem is NP-hard. The Graph
Genus problem has applications in circuit layouts and distributed compu-
tation.
Let G and G0 be two graphs. The bar-amalgamation of G and G0, denoted
G  G0, is the result of running a new edge (called the \bar") from a vertex
of G to a vertex of G0 . The denition of bar-amalgamation on two graphs
can be extended to more than two graphs. Inductively, a bar-amalgamation
of r graphs G1, : : :, Gr , written G1  G2 


 Gr , is the bar-amalgamation
of the graph G1 and the graph G2 


 Gr .
Let G be a graph and let H be a subgraph of G. Let (G) be a rotation
system of G. A rotation system 0(H ) of H can be obtained from (G) by
deleting all edges that are not in H . The rotation system 0 (H ) of H will
be called an induced rotation system of H from the rotation system (G).
The proofs for the following theorem and corollary are omitted.
Theorem 19.7 Let G1,


, Gr be graphs and let (G1 


 Gr ) be a
rotation system of a bar-amalgamation G1 


 Gr of G1 ,


, Gr . Then
X
r
((G1 


 Gr )) = (i(Gi))
i=1
where i (Gi ) is the induced rotation system of Gi from (G1 


 Gr ),
1  i  r.
Corollary 19.8 Let G1,


, Gr be graphs and let G0 be an arbitrary bar-
amalgamation of G1 ,


, Gr . Then
X
r
min(G0) = min(Gi)
i=1

91
 Now we are ready for the main theorem.
Theorem 19.9 For any xed constant , 0   < 1, the Graph Genus
problem cannot be approximated in polynomial time with an absolute dier-
ence n unless P = NP.
proof. Suppose that A is an approximation algorithm that, given a graph
G of n vertices, constructs an embedding of G of genus at most min(G)+ n .
k . Then for suciently large n, we
Let k be an integer such that  < k+1
have n < n k+1 . Thus n(k+1)  nk 1.
k

Let nk G be a graph that is an arbitrary bar amalgamation of nk copies

of G. Then the number of vertices of nk G is N = nk+1 . The graph nk G
can be obviously constructed from G in polynomial time. Moreover, by
Corollary 19.8
min(nk G) = nk
min(G)
Now running the algorithm A on the graph nk G gives us an embedding
(nk G) of nk G, which has genus at most min(nk G) + N  . Therefore,
((nk G))  min(nk G) + N 
= nk min(G) + n(k+1)
 nk min(G) + nk 1 (6)
On the other hand, if we let 1(G),


, nk (G) be the nk induced rotation
systems of G from (nk G), then by Theorem 19.7

k Xnk
((n G)) = (i(G)) (7)
i=1
Combining Equations (6) and (7) and noticing that the genus of i (G) is at
least as large as min(G) for all 1  i  nk , we conclude that at least one
induced rotation system i (G) of G achieves the minimum genus min(G).
This rotation system of G can be found by calculating the genus for each
induced rotation system i (G) from (nk G) and selecting the one with the
smallest genus. This can be accomplished in polynomial time.
Therefore, using the algorithm A, we would be able to construct in poly-
nomial time a minimum genus embedding for the graph G. Consequently,
the Graph Genus problem can be solved in polynomial time. Since the
Graph Genus problem is NP-hard, we would derive P = NP.

92
 The technique of Theorem 19.9 can be summarized as follows. Let Q =
hIQ; SQ; fQ; optQi be an optimization problem such that there is an operator
 implementable in polynomial time that can \compose" input instances,
i.e., for any two input instances x and y of Q, x  y is also an input instance
of Q such that jx  y j = jxj + jy j (in the case of Theorem 19.9,  is the
bar-amalgamation). Moreover, suppose that from a solution sxy to the
instance x  y , we can construct in polynomial time solutions sx and sy for
the instances x and y , respectively such that
fQ(x  y; sxy ) = fQ(x; sx) + fQ(y; sy )
(this corresponds to Theorem 19.7) and
Opt(x  y ) = Opt(x) + Opt(y )
(this corresponds to Corollary 19.8), then using the technique of Theo-
rem 19.9, we can prove that the problem Q cannot be approximated in
polynomial time with a absolute dierence n for any constant  < 1 unless
Q can be solved precisely in polynomial time. In particular, if Q is NP-
hard, then Q cannot be approximated in polynomial time with a absolute
dierence n for any constant  < 1 unless P = NP.
As an easy exercise, readers are advised to use this technique to prove
that the Independent Set problem cannot be approximated in polynomial
time with an absolute dierence n for any constant  < 1.

93
 CPSC-669 Computational Optimization
Lecture #20, October 13, 1995
Lecturer: Professor Jianer Chen
Scribe: Mitrajit Chatterjee
Revision: Jianer Chen
20 Planar Independent Set
The algorithm Knapsack-Dyn for the Knapsack problem and the algo-
rithm c-Scheduling-Dyn for the c-Processor Scheduling problem share
a common property that the algorithms run in time polynomial in length(x)
and max(x) on an input instance x, where length(x) and max(x) are as
dened in Denition 18.1. This motivates the following denition.
Denition 20.1 An algorithm A that solves an optimization problem Q =
hI; S; f; opti is a pseudo-polynomial time algorithm if on any input instance
x 2 I , the running time of A is bounded by a polynomial of length(x) and
max(x). In this case, we say that the optimization problem Q can be solved
in pseudo-polynomial time.
Most fully polynomial time approximation scheme algorithms are derived
from pseudo-polynomial time algorithms for the same problem by properly
scaling and rounding the input data. On the other hand, the following
theorem shows that under a very general condition, the existence of a fully
polynomial time approximation scheme implies the existence of a pseudo-
polynomial time algorithm.
Theorem 20.1 Let Q = hI; S; f; opti be an optimization problem such that
for all input instance x 2 I we have Opt(x)  p(length(x); max(x)), where p
is a two variable polynomial. If Q has a fully polynomial time approximation
scheme, then Q can be solved in pseudo-polynomial time.
proof. Suppose Q is a minimization problem, i.e., opt = min. Since Q has
a fully polynomial time approximation scheme, there is an approximation
algorithm A for Q such that for any input instance x 2 I , the algorithm A
produces a solution y 2 S (x) in time p1 (jxj; 1=) satisfying
f (x; y )  1 + 
Opt(x)
94
where p1 is a two variable polynomial.
In particular, let  = 1=(p(length(x); max(x)) + 1), then the solution y
satises
f (x; y )  Opt(x) + p(length(Opt (x)
x); max(x)) + 1 < Opt(x) + 1
Now since both f (x; y ) and Opt(x) are integers and f (x; y )  Opt(x), we
get immediately f (x; y ) = Opt(x). That is, the solution produced by the
algorithm A is actually an optimal solution. Moreover, the running time of
the algorithm A for producing the solution y is bounded by
p1(jxj; p(length(x); max(x)) + 1)
which is a polynomial of length(x) and max(x). We conclude that the opti-
mization problem Q can be solved in pseudo-polynomial time.
Theorem 17.1 gives a fairly convenient way for checking that an optimiza-
tion problem has no fully polynomial time approximation scheme. How well
can this kind of problems be approximated? In the following, we will show
that for certain problems that have no fully polynomial time approximation
scheme, polynomial time approximation algorithms with approximation ra-
tio 1 +  are still possible, for any xed constant  > 0.
The rst problem to be considered is the Independent Set problem
on planar graphs, dened as follows.
Planar Independent Set (IS)= hI; S; f; opti
I: the set of all planar graphs G
S (G): the collection of all subsets D of the vertices of the graph
G such that no two vertices in D are adjacent
f (G; D): the number of vertices in D
opt: max
It is known that the Planar Independent Set problem is NP-hard.
Moreover, by Theorem 17.1, the Planar Independent Set problem has
no fully polynomial time approximation scheme unless P = NP.
The following theorem by Lipton and Tarjan plays a key role in the
approximation algorithm for the Planar Independent Set problem. The
proof of the theorem is omitted.

95
Theorem 20.2 (Separator Theorem) For any planar graph G = (V; E ),
jV j = n, one can partition the vertex set V of G into three disjoint sets, A,
B, and C , such that
1. jAj; jB j  2n=3;
p
2. jC j  8n; and
3. C separates A and B , i.e. there is no edge between A and B .
Moreover, there is a linear time algorithm that, given a planar graph G,
constructs the triple (A; B; C ) as above.
Let G = (V; E ) be a planar graph and let (A; B; C ) be a triple satisfying
the conditions of Theorem 20.2. We will say that the graph G is split into
two smaller pieces A and B (using the separator C ). A simple observation is
that if DA and DB are independent sets of the graphs induced by the vertex
sets A and B , respectively, then the union DA [ DB is an independent set of
the graph G. Moreover, since the sizes of the sets pA and B are of order
(n)
while the size of the separator C is of order O( n), ignoring the vertices
in the separator C does not seem to lose too much precision. This idea
is implemented by the following algorithm, where K is a constant to be
determined later.
Algorithm 20.1 PlanarIndSet(K )
Input: a planar graph =( G V; E )
Output: an independent Set S in G
1. If (jV j  K ) then
find a maximum indepenent set in Dusing G
exhaustive search;
Return( ); D
fAt this point jV j > K g
.
2. split V
into ( A; B; C
) as in Theorem 20.2;
3. recursively find an independent set A for D and an A
independent set D
B for ; B
4. return( A D [DB ) .

By the discussion above, the algorithm PlanarIndSet correctly returns

an independent set for the graph G. Thus, it is an approximation algorithm

96
for the Planar Independent Set problem. We rst study a few properties
of this algorithm.
The algorithm splits the graph G into small pieces. If the size of a piece
is larger than K , then the algorithm splits the piece into two smaller pieces
in linear time according to Theorem 20.2. Otherwise, it nds a maximum
independent set for the piece using brute force method. We rst discuss the
number of pieces whose size is within a certain region.
A piece is at level 0 if its size is not larger than K . For a general i  0,
a piece is at level i if its size (i.e., the number of vertices in the piece) is in
the region ((3=2)i 1K; (3=2)iK ], i.e., if its size is larger than (3=2)i 1K but
not larger than (3=2)iK . Note that the largest level number is bounded by
log(n=K )= log(3=2) = O(log(n=K )).
Lemma 20.3 For a xed i, each vertex of the graph G belongs to at most
one piece at level i.
proof. Fix a vertex v of the graph G.
Suppose that the largest level number is h and that the graph G is at
level h. By the denition, n  (3=2)hK . Now according to Theorem 20.2,
G is split into two pieces A and B , whose size is bounded by
2n=3  (2=3)(3=2)hK = (3=2)h 1K
Thus, both pieces A and B do not belong to level h. Consequently, G is the
only piece at level h. Thus, there is only one piece at level h that contains
the vertex v .
Inductively, suppose that for each i  j , at most one piece at level i
contains the vertex v and there is a piece P at level j that contains the
vertex v . If j = 0, then we are done. Otherwise, let P1 and P2 be the two
smaller pieces obtained by splitting the piece P according to Theorem 20.2.
As we proved above for level h, no P1 and P2 can be at level j . Moreover, at
most one of P1 and P2 can contain the vertex v . Without loss of generality,
suppose that P1 contains v and that P1 is at level j 0 < j . Now for each
i  j 0, at most one piece at level i contains the vertex v. The induction goes
through.
Therefore, all pieces at level i are disjoint. Since each piece at level i con-
sists of more than (3=2)i 1K vertices, there are no more than (2=3)i 1(n=K )
pieces at level i, for all i. We summarize these discussions as follows.

97
  There are no more than n pieces at level 0, each is of size at most K ;
 For each xed i > 0, there are no more than (2=3)i 1(n=K ) pieces at
level i, each is of size bounded by (3=2)iK ; and
 There are at most O(log n) levels.
Now we are ready to analyze the algorithm.
Lemma 20.4 The running time of the algorithm PlanarIndSet is bounded
by O(n log n + 2K n).
proof. For each piece at level i > 0, we apply Theorem 20.2 to split
it into two smaller pieces in time linear to the size of the piece. Since the
total number of vertices belonging to pieces at level i is bounded by n, we
conclude that the total time spent by the algorithm PlanarIndSet on pieces
at level i is bounded by O(n) for each i > 0. Since there are only O(log n)
levels, the algorithm PlanarIndSet takes time O(n log n) on piece splitting.
For each piece P at level 0, which has size bounded by K , the algorithm
nds a maximum independent set by checking all subsets of vertices of the
piece P . There are at most 2K such subsets in P , and each such a subset
can be checked in time linear to the size of the piece. We conclude that
the algorithm PlanarIndSet spends time O(2K n) on pieces at level 0. In
summary, the running time of the algorithm PlanarIndSet is bounded by
O(n log n + 2K n).
Let us consider the approximation ratio for the algorithm PlanarIndSet.
Fix an i > 0. Suppose that we have l pieces of size n1 , n2 , : : :, nl at
level i. For each such a piece of size nq , a separator of size less than 3pnq
is constructed to split the piece into two smaller pieces. The vertices in the
separator will be ignored in the further consideration. There are at most
3pn1 + 3pn2 +


+ 3pnl
vertices that belong to separators for pieces at level i. It is well-known that
the above summation will be maximized when all n1 , n2 , : : :, nl are equal.
As n1 + n2 +


+ nl  n, each nq can be at most (n=l). Hence, the above
summation is bounded by
q q q q p
3 n=l + 3 n=l +


+ 3 n=l = 3l n=l = 3 nl
| {z }
l terms
98
Now, since l  (2=3)i 1(n=K ) (as derived above), the total number of ver-
tices belonging to separators for pieces at level i is bounded by
s 2 i 1 n 3n  2  i 1
2
3 n 3 K=p 3 K
Let F denote the set of all vertices that belong to a separator at some
level. We derive
h  3n  r 2 !i 1  3n  X
X 1 r 2 !i 1 3nd
jF j  p 3  p 3 =p
i=1 K K i=1 K
p i1
where d = P1
i=1 ( 2=3) is a constant.
Lemma 20.5 Let S be the solution produced by the algorithm PlanarIndSet.
Then Opt(G)  jS j + jF j.
proof. Let P be a piece at level 0 and let Smax be a maximum independent
set of the graph G. It is easy to see that Smax \ P is an independent set
in the piece P , which cannot be larger than the maximum independent set
P of P constructed by the algorithm PlanarIndSet. Note that S is the
Smax
P over all pieces at level 0. We have
union of Smax
[
Smax = (Smax \ P ) [ (Smax \ F )
P : level 0 piece
Therefore,
X
Opt(G) = jSmaxj  (jSmax \ P j) + jSmax \ F j
XP : level 0P piece
 jSmaxj + jF j
P : level 0 piece
= jS j + jF j
The lemma is proved.
From Lemma 20.5, we get immediately
Opt(G)  1 + jF j
jS j Opt(G) jF j
99
Since the graph G is planar, by the famous Four-Color Theorem, G can be
colored with at most 4 colors such that no two adjacent vertices in G are of
the same color. It is easy to see that all vertices with the same color form an
independent set for G. We conclude that the size Opt(G) of the maximum
independent set Smax of G is at least a quarterpof the size n of the graph G.
Combining Opt(G)  n=4 with jF j  3nd= K , we obtain
Opt(G)  1 + jF j jF j
jS j Opt(G) pjF j  1 + (n=4) jF j
 1 + 3nd= K p = 1 + p 12d
(n=4) 3nd= K K 12d
Now for any xed constant , if we let
K = (12d(1 + 1=))2 = 144d2(1 + 1=)2
then the algorithm PlanarIndSet(K ) produces an independent set S for G
with approximation ratio
Opt(G)  1 + 
jS j
in time O(n log n + n2144d2(1+1=)2 ). For a xed  > 0, this is a polynomial
time algorithm. However, this is not a fully polynomial time approximation
scheme since its time complexity is not bounded by a polynomial of n and
1=.

100
 CPSC-669 Computational Optimization
Lecture #21, October 16, 1995
Lecturer: Professor Jianer Chen
Scribe: Mitrajit Chatterjee
Revision: Jianer Chen
21
-TSP: rst algorithm
The approximation algorithm PlanarIndSet for the Planar Independent
Set problem motivates the following denition.
Denition 21.1 An optimization problem Q has a polynomial time approx-
imation scheme (PTAS), if for any xed constant  > 0, there is a polynomial
time approximation algorithm for Q with approximation ratio bounded by
1 + .
Note that a polynomial time approximation scheme does not require the
running time of the approximation algorithm to be bounded by a polynomial
of 1=.
The previous lecture shows that the Planar Independent Set prob-
lem has a polynomial time approximation scheme. According to Theo-
rem 17.1, the Planar Independent Set problem has no fully polyno-
mial time approximation scheme unless P = NP. Thus, a polynomial time
approximation scheme seems the best we can hope for the problem.
Other optimization problems that have polynomial time approximation
schemes but have no fully polynomial time approximation schemes include
the Planar Vertex Cover problem and some other optimization prob-
lems on planar graphs. Most of these polynomial time approximation scheme
algorithms use the similar technique as the one we described for the Planar
Independent Set problem, i.e., using Separator Theorem (Theorem 20.2)
to separate a planar graph into small pieces by separators of small size and
using brute force method to solve the problem for the small pieces. Stu-
dents are encouraged to apply this technique to derive a polynomial time
approximation scheme for the Planar Vertex Cover problem.
A dierence separating technique has been proposed by Baker (1994)
to derive polynomial time approximation scheme for optimization problems
on planar graphs. We brie y describe the idea here based on the Planar

101
Independent Set. Let G be a planar graph. Embed G into the plane. Now
the vertices on the unbounded face of the embedding give the rst layer of
the graph G. By peeling the rst layer, i.e., deleting the vertices in the rst
layer, we obtain (maybe more than one) several separated pieces, each of
which is a planar graph embedded in the plane. Now the rst layers of these
pieces form the second layer for the graph G. By peeling the second layer of
G, we obtain the third layer, and so on. Dene depth of the planar graph G
to be the maximum number of layers of the graph. Baker observed that for
a graph of constant depth, a maximum independent set can be constructed
in polynomial time by dynamic programming. Moreover, for any graph G of
arbitrary depth, if we remove one layer out of every K consecutive layers, we
obtain a set of separated planar graphs of constant depth. Now for each such
graph of constant depth, we construct a maximum independent set. The
union of these maximum independent sets forms an independent set for the
original graph G. For suciently large K , the number of vertices belonging
to the removed layers is very small and thus gives only a small error in
the approximation. Baker demonstrated a polynomial time approximation
scheme for the Planar Independent Set problem with running time
bounded by O(81=n=).
Another optimization problem that has a polynomial time approxima-
tion scheme but has no fully polynomial time approximation scheme is the
Multi-Processor Scheduling problem. The polynomial time approxi-
mation scheme algorithm for this problem is closely related to approxima-
tion algorithms for the Bin Packing problem. We will discuss this after
the study of approximation algorithms for the Bin Packing problem.
We will study in this lecture a restricted version of the Traveling
Salesman problem.
Denition 21.2 Let G = (V; E ) be a weighted, undirected, and complete
graph. We say that the graph G satises the triangle inequality if for any
three vertices u, v , and w of G we have
weight(u; w)  weight(u; v ) + weight(v; w)
The Traveling Salesman problem under triangle inequality is dened
as follows.

-Traveling Salesman Problem (
-TSP)
Input: a weighted, undirected, and complete graph G
satisfying the triangle inequality
102
 Output: a simple cycle of minimum weight that contains
all vertices of G
Let G be an input instance of the
-TSP. Every solution to G, i.e., every
simple cycle in G that contains all vertices of G, will be called a traveling
salesman tour.
An important case of the
-TSP is the Euclidean TSP, in which each
vertex is a point in the Euclidean plane and the weight of an edge (w; u)
equals the Euclidean distance between w and u.
Remark 21.3 Both
-TSP and Euclidean TSP are NP-hard.
We present the rst approximation algorithm for the
-TSP based on
minimum spanning trees.
Algorithm 21.1 EasyTSP
Input: an input instance G
of
-TSP
Output: a traveling salesman tour L
1. construct a minimum spanning tree for T
; G
2. perform a depth first search on the tree T
to compute
the dfs number for each vertex of ; T
3. let Lbe the list of vertices of T
sorted by their dfs
numbers;
4. return L
as a traveling salesman tour for . G
The analysis of the time complexity of the above algorithm is pretty
simple. It is well-known that a minimum spanning tree of a graph of n
vertices can be constructed in time O(n2 ). It is also easy to see that each of
the steps 2, 3, and 4 takes time bounded by O(n2). Therefore, the algorithm
EasyTSP runs in time O(n2 ).
Now we consider the approximation ratio for the algorithm. A depth
rst search process that computes the dfs numbers for vertices of a tree can
be implemented by the following simple algorithm.
Algorithm 21.2 DFS(v)
1. counter = counter + 1;
2. v
dfs[ ] = counter;
3. for each child w
of v do
w
DFS( );

103
 a
b s 
  sXXXXsc
d s s
XXXXsf
e
g s

Figure 6: The minimum spanning tree T

This recursive subroutine is called by the following main program.
Algorithm 21.3 Main
fsuppose that vertex 1 is the root of the tree Tg
1. counter = 0;
2. DFS(1).

The depth rst search process on the tree T can be regarded as a closed
walk L0 of the tree (a closed walk is a cycle in T in which each vertex may
appear more than once). Each edge (u; v ), where u is the father of v in T ,
is traversed exactly twice in the walk L0 : the rst time when DFS(u) calls
DFS(v ) we traverse the edge from u to v , and the second time when DFS(v )
is nished and returns back to DFS(u) we traverse the edge from v to u.
Therefore, the walk L0 has weight exactly twice the weight of the tree T . It
is also easy to see that the list L produced by the algorithm EasyTSP can
be obtained from the walk L0 by deleting for each vertex v all but the rst
occurrences of v in the list L0. Since each vertex appears exactly once in
the list L and G is a complete graph, L corresponds to a traveling salesman
tour.
Example 21.4 Consider the tree T in Figure 6, where a is the root of the
tree T . The depth rst search process traverses the tree T in the order
a; b; d; b; a; c; e; c; f; g; f; c; a
By deleting for each vertex v all but the rst vertex occurrences for v , we
obtain the list of vertices of the tree T sorted by their dfs numbers
a; b; d; c; e; f; g
Deleting a vertex occurrence of v in the list f


uvw


g is equivalent
to replacing the path u ! v ! w by a single edge (u; w). Since the graph
G satises the triangle inequality, deleting vertex occurrences from a walk
104
does not increase the weight of the walk. Consequently, the weight of the
traveling salesman tour L is not larger than the weight of the closed walk
L0, which is bounded by 2 times the weight of the minimum spanning tree
T.
Note that for the Traveling Salesman problem, we can assume with-
out loss of generality that all edge weights are non-negative integers (oth-
erwise we add a suciently large weight to each edge). Observe that the
weight of any traveling salesman tour is at least as large as the weight of the
minimum spanning tree T | removing any edge (of non-negative weight)
from the traveling salesman tour results in a spanning tree of the graph G.
In conclusion, the traveling salesman tour L constructed by the algorithm
EasyTSP has weight bounded by 2 times the weight of a minimum traveling
salesman tour. We conclude with the following theorem.
Theorem 21.1 The approximation ratio of the algorithm EasyTSP is bounded
by 2.
We give a simple example to show that the ratio 2 is tight for the ap-
proximation algorithm EasyTSP in the sense that there are input instances
for the
-TSP for which the algorithm EasyTSP produces a solution with
approximation ratio arbitrarily close to 2. This kind of input instances can
actually appear for the Euclidean TSP. Consider the gures in Figure 7.
Suppose we are given 2n points on the Euclidean plane with polar coordi-
nates xk = (b; 2k=n) and yk = (b + d; 2k=n), k = 1; : : :; n, where d is
much smaller than b. See Figure 7(a), where n = 8. Then it is not hard (for
example, by Kruskal's algorithm for minimum spanning tree) to see that the
edges (xk ; xk+1 ), k = 1; : : :; n 1 and (xj ; yj ), j = 1; : : :; n form a minimum
spanning tree T for the set of points. See Figure 7(b). Now if we perform a
depth rst search on T starting from the vertex x1 and construct a traveling
salesman tour, we will get a tour L that is shown in Figure 7(c) while an
optimal traveling salesman tour L0 is shown in Figure 7(d).
The weight of the tour L is about 2a(n 1)+2d, where a is the distance
between two adjacent points xk and xk+1 (note that when d is suciently
small compared with a, the distance between two adjacent points yk and
yk+1 is roughly equal to the distance between the two corresponding points
xk and xk+1 ), while the optimal traveling salesman tour has weight roughly
nd + na. When d is suciently small compared with a and when n is
suciently large, the ratio of the weight of the tour L and the weight of the
tour L0 can be arbitrarily close to 2.

105
 y3 y3
y4 y2 y4 y2
a
x3 x3 d
x4 x2 x4 x2
b d
y5 x5 y1 y5 x5 y1
x1 x1
x 6 x x8 x 6 x x8
7 7

y6 y8 y6 y8
y7 y7

(a) (b)

y3 y3
y4 y2 y4 y2
x3 x3
x4 x2 x4 x2
y5 x5 y1 y5 x5 y1
x1 x1
x 6 x x8 x6 x8
7 x7

y6 y8 y6 y8
y7 y7

(c) (d)
Figure 7:
-TSP Example.

106
 CPSC-669 Computational Optimization
Lecture #22, October 18, 1995
Lecturer: Professor Jianer Chen
Scribe: Mitrajit Chatterjee
Revision: Jianer Chen
22
-TSP: Christodes algorithm
In this lecture, we will allow a graph to have \multiple edges", i.e., each pair
of vertices of a graph can be connected by more than one edge.
Let us reconsider the approximation algorithm EasyTSP for the
-TSP
problem. As we pointed out, after the minimum spanning tree T is con-
structed, we traverse the tree T by a depth rst search process in which
each edge of T is traversed exactly twice. This process can be re-interpreted
as follows:
1. construct a minimum spanning tree;
2. double each edge of T into two edges, each of which has the same
weight as the original edge. Let the resulting graph be D;
3. make a closed walk W in the graph D such that each edge of D is
traversed exactly once in W ;
4. use \shortcuts", i.e., delete all but the rst occurrences for each vertex
in the walk W to make a traveling salesman tour L.
There are three crucial facts that make the above algorithm correctly
produce a traveling salesman tour with approximation ratio 2: (1) the graph
D gives a closed walk in the graph G and D contains all vertices of G; (2)
the total weight of the graph D is bounded by 2 times the weight of an
optimal traveling salesman tour; and (3) the shortcuts do not increase the
weight of a closed walk so that we can derive a traveling salesman tour L
from D without increasing the weight of the walk.
Therefore, if we can construct a better graph D1 whose weight is smaller
than the graph D constructed by the algorithm EasyTSP such that D1 forms
a closed walk of G and that D1 contains all vertices of G, then using the
shortcuts on D1 should derive a better approximation to the minimum trav-
eling salesman tour.
For this, we need introduce a denition.
107
Denition 22.1 An undirected connected graph G is an Euler graph if
there is a closed walk in G that traverses each edge of G exactly once.
Recent research has shown that Euler graphs play an important role in
designing ecient parallel graph algorithms.
Theorem 22.1 An undirected connected graph G is an Euler graph if and
only if every vertex of G has an even degree.
proof. Suppose that G is an Euler graph. Let W be a closed walk in G
that traverses each edge of G exactly once.
Let v be a vertex of G. Since W is a closed walk, each time W enters the
vertex v from an edge, W must leave the vertex v by another edge incident
on v . Therefore, each edge incident on v that is an \incoming" edge for W
must be paired with an edge incident on v that is an \outgoing" edge for
W . Since W traverses each edge exactly once, we conclude that the number
of edges incident on v , i.e., the degree of v , is even.
Conversely, suppose that all vertices of the graph G have even degree.
We prove the theorem by induction on the number of edges in G. The
minimum such a graph G in which all vertices have even degree consists of
two vertices connected by two (multiple) edges. This graph is clearly an
Euler graph.
Now suppose that G has more than two edges. Let v0 be any vertex of
G. We construct a maximal walk W0 starting from the vertex v0 . That is,
starting from v0 , on each vertex if there is an unused edge, then we extend
W0 along that edge (if there are more than one such edge, we pick any
one). The process stops when we hit a vertex u on which there is no unused
incident edge. We claim that the ending vertex u must be the starting vertex
v0 . In fact, for each interior vertex w in the walk W0, each time W0 passes
through, W0 uses one edge to enter w and uses another edge to leave w.
Therefore, if the process stops at u and u 6= v0 , then the walk W0 has only
used an odd number of edges incident on u. This contradicts our assumption
that the vertex u is of even degree. This proves the claim. Consequently,
the walk W0 is a closed walk.
The closed walk W0 can also be regarded as a graph. By the denition,
the graph W0 itself is an Euler graph. According to the proof for the rst
part of this theorem, all vertices of the graph W0 have even degree. Now
removing all edges in the walk W0 from the graph G results in a graph
G0 = G W0. The graph G0 may not be connected. However, all vertices

108
of G0 must have an even degree because each vertex of the graphs G and
W0 has an even degree.
Let C1, C2, : : :, Ch be the connected components of the graph G0 . By
the inductive hypothesis, each connected component Ci is an Euler graph.
Let Wi be a closed walk in Ci that traverses each edge of Ci exactly once,
for i = 1; : : :; h. Moreover, for each i, the closed walk W0 contains at least
one vertex vi in the connected component Ci (if W0 does not contain any
vertex from Ci , then the vertices of Ci have no connection to the vertices in
the walk W0 in the original graph G, this contradicts the assumption that
the graph G is connected).
Therefore, it is easy to insert each closed walk Wi into the closed walk
W0 (by replacing any vertex occurrence of vi in W0 by the list Wi, where Wi
is given by beginning and ending with vi ), for all i = 1; : : :; h. This forms
a closed walk W for the original graph G such that the walk W traverses
each edge of G exactly once. Thus, the graph G is an Euler graph.
The proof of Theorem 22.1 suggests an algorithm that constructs a closed
walk W for an Euler graph G such that the walk W traverses each edge
of G exactly once. This walk will be called an Euler tour. By a careful
implementation, one can make this algorithm run in linear time. We leave
the detailed implementation to the reader. Instead, we state this result
without a proof as follows.
Theorem 22.2 There is an algorithm that, given an Euler graph, con-
structs an Euler tour in linear time.
Now we are ready to show how a better Euler graph D1 can be con-
structed based on a minimum spanning tree, from which a better approxi-
mation for the minimum traveling salesman tour can be derived.
Let G be an input instance of the
-TSP problem and let T be a mini-
mum spanning tree in G. We have
Lemma 22.3 The number of vertices of the tree T that has an odd degree
in T is even.
proof. Let v1 , : : :, vn be the vertices of the tree T . Since each edge
e = (vi; vj ) of T contributes one degree to vi and one degree to vj , we must
have
X
n
degT (vi) = 2(n 1)
i=1

109
where degT (vi ) is the degree of the vertex vi in the tree T . Note that n 1
is the number of edges in the tree T . We partition the set of vertices of T
into odd degree vertices and even degree vertices. Then we have
X X
degT (vi ) + degT (vj ) = 2(n 1)
vi: even degree vj : odd degree
P
both vi : even degree degT (vi ) and 2(n 1) are even numbers, the
Since P
value vj : odd degree degT (vj ) is also an even number. Consequently, the
number of vertices that have odd degree in T must be even.
By Lemma 22.3, we can suppose, without loss of generality, that v1, v2 ,
: : :, v2h be the odd degree vertices in the tree T . The vertices v1, v2, : : :, v2h
induce a complete subgraph H in the original graph G. Now construct a
minimum weight complete matching Eh in H . The matching Eh consists of
h edges such that each of the vertices v1, v2 , : : :, v2h is incident on exactly
one edge in Eh . Thus, adding the edges in Eh to the tree T results in a graph
D1 = T +Eh in which all vertices have an even degree. By Theorem 22.1, the
graph D1 is an Euler graph. Moreover, the graph D1 contains all vertices of
the graph G. We are now able to derive a traveling salesman tour L1 from
D1 by using shortcuts.
We formally present this in the following algorithm. The algorithm is
due to N. Christodes.
Algorithm 22.1 Christofides
Input: an input instance of
-TSP G
Output: a traveling salesman tour L
1. construct a minimum spanning tree for ; T G
2. v
let 1, ::: v
, 2h be the odd degree vertices in , T
construct a minimum weight matching h in the E
complete graph induced by 1, , 2h ; v ::: v
3. construct an Euler tour W
1 in the Euler graph
D T E
1 = + h;
4. use shortcuts to derive a traveling salesman tour L1
from W
1;
5. return 1 .L
It is known that a minimum weight matching can be constructed in time
O(n3) (see lecture notes 7-10 for discussion on graph matchings). The other
110
steps of the algorithm Christofides clearly take time O(n3 ). Therefore,
the algorithm Christofides runs in time O(n3 ).
Now let us study the approximation ratio for the algorithm Christofides.
LemmaP22.4 The weight of the minimum weight matching Eh on v1, v2,
: : :, v2h, e2Eh weight(e), is at most 1=2 of the weight of an optimal traveling
salesman tour in the graph G.
proof. Let L be an optimal traveling salesman tour in the graph G. By
using shortcuts, i.e., by removing the vertices that are not in fv1 ; v2; : : :; v2hg
from the tour L, we obtain a simple cycle L0 that contains exactly the vertices
v1 , : : :, v2h. Since G satises the triangle inequality, the weight of L0 is not
larger than the weight of L.
Moreover, the simple cycle L0 can be decomposed into two disjoint
matchings of fv1; : : :; v2hg | one matching is obtained by taking every
other edge in the cycle L, and the other matching is formed by the rest
of the edges. Of course, both of these two matchings have weight at least as
large as the minimum weight matching Eh on fv1 ; : : :; v2h g. This gives
weight(L)  weight(L0 )  2
weight(Eh )
This completes the proof.
Now the analysis is clear. We have D1 = T + Eh . Thus
weight(D1) = weight(T ) + weight(Eh )
By the analysis for the algorithm EasyTSP (Algorithm 21.1), the weight of
T is not larger than the weight of an optimal traveling salesman tour for G.
Combining this with Lemma 22.4, we conclude that the weight of the graph
D1 is bounded by 1:5 times the weight of an optimal traveling salesman
tour in G. Moreover, the traveling salesman tour L1 constructed by the
algorithm Christofides is obtained by using shortcuts on the graph D1
and thus has weight not larger than the weight of D1. We close this lecture
with the following theorem.
Theorem 22.5 The algorithm Christofides for the
-TSP problem runs
in time O(n3 ) and has approximation ratio 1:5.
As for the algorithm EasyTSP, one can show that the ratio 1:5 is tight for
the algorithm Christofides in the sense that there are input instances of

-TSP for which the algorithm Christofides produces traveling salesman
tours with approximation ratio arbitrarily close to 1:5.

111
 CPSC-669 Computational Optimization
Lecture #23, October 20, 1995
Lecturer: Professor Jianer Chen
Scribe: Weijie Zhang
Revision: Jianer Chen
23 Bin Packing problem
In the previous lectures, we have presented several approximation algorithms
for NP-hard optimization problems. In this lecture, we study approximation
algorithms for the Bin Packing problem. Recall that the Bin Packing
problem is dened as
Bin Packing
Input: ht1 ; t2; : : :; tn ; B i, all integers and ti  B for all i
Output: a packing of the n objects of size t1 , : : :, tn into the
minimum number of bins of size B
Since the number of bins used by any packing cannot be larger than the
number of objects in the input, according to Theorem 17.1, the Bin Pack-
ing problem has no fully polynomial time approximation scheme. On the
other hand, it is fairly easy to design a polynomial time approximation algo-
rithm for the Bin Packing problem with a reasonably good approximation
ratio. Consider the following simple approximation algorithm for the Bin
Packing problem.
Algorithm 23.1 First-Fit (FF)
Input: I = ht1; t2;


; tn ; Bi
Output: a packing of the n objects into bins of size B
1. for i = 1 to n do
2. j=1
3. notput = true;
4. while notput do
5. if object ican be put in bin j
6. then put i j
in bin ; notput = false;
7. else j = j + 1
112
 The for loop in the algorithm is executed n times, and in each execution,
the while loop can be done in O(n) time since ti  B for all i. This
concludes that the algorithm First-Fit runs in time O(n2 ). What is the
approximation ratio for the algorithm?
Theorem 23.1 The algorithm First-Fit has approximation ratio 2.
proof. We observe that there is at most one used bin whose content
is not larger than B=2. In fact, suppose that there are two used bins Bi
and Bj whose contents are bounded by B=2. Without loss of generality, let
i < j . Then the algorithm First-Fit would have put the objects in the bin
Bj into the bin Bi since the bin Bi has enough room for them and the bin
Bi is considered before the bin Bj by the algorithm First-Fit.
Now the theorem can be proved in two cases.
Suppose that the contents of all used bins are not less than B=2. Let m
be the number of bins used by the algorithm First-Fit. We have
X
n
ti  mB
2
i=1
Since the bin size is B , we need at least
X
n
d( ti)=Be  d(mB)=(2B)e  m=2
i=1
bins to pack the n objects, i.e., Opt(I )  m=2. Therefore, the approximation
ratio is bounded in this case by
m m
Opt(I )  m=2 = 2
Now suppose that there is a used bin whose content x is less than B=2.
Again let m be the number of bins used by the algorithm First-Fit. There-
fore, there are m 1 bins with contents at least B=2. This gives us
X
n
ti  (m 2 1)B + x > (m 2 1)B
i=1
P
Thus, d( ni=1 ti )=B e > (m 1)=2

113
 P
If m 1 is an even number, then since both d( ni=1 ti )=B e and (m 1)=2
are integers, we get
X
n
d( ti )=Be  (m 1)=2 + 1 > m=2
i=1
If m 1 is an odd number, then
Xn
d( ti )=Be  d(m 1)=2e = (m 1)=2 + 1=2 = m=2
i=1
P
Note that any packing should use at least d( ni=1 ti )=B e bins. In partic-
ular,
Xn
Opt(I )  d( ti )=B e
i=1
The above analysis shows that the approximation ratio is bounded by
m  P m  m =2
n
Opt(I ) d( i=1 ti )=B e m=2
This proves the theorem.
Therefore, the Bin Packing problem can be approximated in polyno-
mial time with approximation ratio 2. Can we do better than 2? In particu-
lar, does the Bin Packing problem have a polynomial time approximation
scheme? A negative answer to this question can be easily derived, as shown
in the following theorem.
Theorem 23.2 There is no polynomial time approximation algorithm for
the Bin Packing problem with approximation ratio less than 1.5 unless
P = NP .
proof. Suppose that we have a polynomial time approximation algorithm
A with approximation ratio less than 1.5 for the Bin Packing problem.
We show how we can use this algorithm to solve in polynomial time the
Partition problem, which is NP-complete.
Recall that Partition is a decision problem dened as follows.
Partition
Input: A set fx1 ; x2; : : :; xn g of n integers
Question: Is there
P a0 xsubset P S 0  S such that
i2S i = j 2S S 0 xj ?

114
 PGiven an input instance X = ft1; t2;


; tn g for the Partition problem,
if ni=1 tiPis an odd number, then we know X is a NO-instance. Otherwise,
let B = ( ni=1 ti )=2, and let g (X ) = ht1 ; t2;


; tn ; B i be an instance for the
problem Bin Packing. Now apply the approximation algorithm A for the
Bin Packing problem on the input g (X ). Suppose that the approximation
algorithm A uses m bins for this input instance g (X ). There are two dierent
cases.
If m  3 bins, then since we have
m=Opt(g (X )) < 1:5
we get Opt(g (X )) > 2. ThatPis, the objects t1 , : : :, tn cannot be packed
into two bins of size B = ( ni=1 ti )=2. Consequently, the instance X =
ft1; t2;


; tng is a NO-instance for the Partition problem.
On the other hand, if m  2, then we must have m = 2. Thus, the
objects t1 , : : :, th can be evenly split into two sets of equal size. That is, the
instance X = ft1 ; t2;


; tn g is a YES-instance for the Partition problem.
Therefore, the instance X is a YES-instance for the Partition problem
if and only if the approximation algorithm A uses two bins to pack the
instance g (X ). Since by our assumption, the approximation algorithm A
runs in polynomial time, we conclude that the Partition problem can be
solved in polynomial time.
Since the Partition problem is NP-complete, this implies P = NP. The
theorem is proved.
We observe that the 1:5 lower bound on approximation ratio for the
Bin Packing problem occurs when the optimal value Opt(X ) is very small.
Similar lower bounds on approximation ratio can be derived for optimiza-
tion problems that remain NP-hard even when the optimal value is very
small. Examples include Graph Coloring and Graph Edge Coloring
problems.
In some cases, we may be interested in the asymptotic lower bounds on
approximation ratio of an optimization problem. For instance, we may want
to ask whether the 1:5 lower bound can still be achieved when the optimal
value is suciently large for a input instance for the Bin Packing problem.
This question is closely related to the concept of asymptotic approximation
scheme dened as follows.
Denition 23.1 An optimization problem Q = hI; S; f; opti has a asymp-
totic polynomial time approximation scheme (APTAS) if for any xed con-
stant  > 0, there is a constant c and a polynomial time approximation
115
algorithm A for Q such that for all input instances x 2 I with Opt(x)  c ,
the algorithm A produces a solution for x with approximation ratio bounded
by 1 + .
We will show that the Bin Packing problem has an asymptotic poly-
nomial time approximation scheme.
Let us start with a restricted version of the Bin Packing problem, which
will be called the (;  )-Bin Packing problem. There are two restrictions.
First, we assume that the input objects have at most a constant number
 of dierent sizes. Second, we assume that the size of each input object
is at least as large as a  factor of the bin size. The following is a formal
denition.
(;  )-Bin Packing
Input: ht1 : n1 ; t2 : n2 ; : : :; t : nP
; B i, where B  ti  B for
all i, interpreted as: for the n = i=1 ni input objects, ni of
them are of size ti , for i = 1; : : :; 
Output: a packing of the n objects into the minimum number
of bins of size B
We rst study the properties of the (;  )-Bin Packing problem. Let
I = ht1 : n1; : : :; t : n ; B i be an input instance for the (; )-Bin Packing
problem. Suppose that an optimal packing packs the objects in I into m
bins B1 , B2 , : : :, Bm . Consider the rst bin B1 . Suppose that the bin B1
contains b1 objects of size t1 , b2 objects of size t2 , : : :, and b objects of size
t . We then call
(b1; b2; : : :; b )
the conguration of the bin B1 . Since each object has size at least B and
the bin size is B , the bin B1 contains at most 1= objects. In particular,
we have bi  1= for all i. Therefore, the total number of dierent bin
congurations is bounded by (1= ) .
Now consider the set I 0 of objects that is obtained from the set I with
all objects packed in the bin B1 removed. The set I 0 can be written as
I 0 = ht1 : (n1 b1); t2 : (n2 b2 ); : : :; t : (n b ); B i
Note that I 0 is also an input instance for the (;  )-Bin Packing problem
and the packing (B2 ; B3 ; : : :; Bm ) is an optimal packing for I 0 (I 0 cannot be
packed into less than m 1 bins otherwise the set I can be packed into less
116
than m bins). Therefore, if we can pack the set I 0 into a minimum number
of bins then an optimal packing for the set I can be obtained by packing
the rest of the objects into a single bin B1 .
Now the problem is that we do not know the conguration for the bin
B1. Therefore, we will try all possible congurations for a single bin, and
recursively nd an optimal packing for the rest of the objects. As pointed
out above, the number of bin congurations is bounded by (1= ) , which is
a constant when both  and  are xed. In the next lecture, we will present
a dynamic programming algorithm that constructs an optimal packing for
an input instance for the (;  )-Bin Packing problem.

117
 CPSC-669 Computational Optimization
Lecture #24, October 23, 1995
Lecturer: Professor Jianer Chen
Scribe: Weijie Zhang
Revision: Jianer Chen
24 The (; )-Bin Packing problem
Recall that the (;  )-Bin Packing problem is dened as follows:
(;  )-Bin Packing
Input: ht1 : n1 ; t2 : n2 ; : : :; t : nP
; B i, where B  ti  B for
all i, interpreted as: for the n = i=1 ni input objects, ni of
them are of size ti , for i = 1; : : :; 
Output: a packing of the n objects into the minimum number
of bins of size B
Fix an input instance I = ht1 : n1 ; : : :; t : n ; B i of the (;  )-Bin
Packing problem. Each subset of objects in I can be written as a  -tuple
[h1 ; : : :; h ] with hi  ni to specify that the subset contains hi objects of
size ti for all i. In particular, the input instance I itself can be written as
[n1 ; : : :; n ].
Let #H [h1; : : :; h ] denote the minimum number of bins needed to pack
the subset [h1; : : :; h ] of the input instance I for the (;  )-Bin Pack-
ing problem. Suppose that #H [h1; : : :; h ]  1. According to the discus-
sion in the last lecture, we know that #H [h1; : : :; h ] is equal to 1 plus
#H [h1 b1; : : :; h b ] for some bin conguration (b1; b2; : : :; b ). On the
other hand, since #H [h1; : : :; h ] corresponds to an optimal packing of the
subset [h1; : : :; h ], #H [h1; : : :; h ] is actually equal to 1 plus the minimum
of #H [h1 b1; : : :; h b ] over all consistent bin congurations (b1; : : :; b ).
This suggests an algorithm that uses the dynamic programming technique
to compute the value of #H [h1; : : :; h ]. In particular, #H [n1; : : :; n ] gives
the optimal value for the input instance I for the (;  )-Bin Packing prob-
lem.
Denition 24.1 Fix an input instance I = ht1 : n1; : : :; t : n ; Bi for the
(;  )-Bin Packing problem. Let I 0 = [h1 ; : : :; h ] be a subset of the input
118
objects in I , where hi  ni for all i. A  -tuple (b1; : : :; b ) is an addable bin
conguration to I 0 if
1. hPi + bi  ni for all i = 1; : : :;  ; and
2. i=1 ti bi  B .
Intuitively, an addable bin conguration species a bin conguration
that can be obtained using the objects in I that are not in the subset I 0.
Now we are ready for presenting the following dynamic programming
algorithm. We use a  -dimensional array H [1::n1; : : :; 1::n ] (note that  is
a xed constant) such that H [i1; : : :; i ] records an optimal packing for the
subset [i1; : : :; i ] of I . We use the notation #H [i1; : : :; i ] to denote the
number of bins used in the packing H [i1; : : :; i ]. For a packing H [i1; : : :; i ]
and a bin conguration (b1; : : :; b ), we will use
H [i1; : : :; i ]  (b1; : : :; b)
to represent the packing for the subset [i1 + b1; : : :; i + b ] that is obtained
from H [i1; : : :; i ] by adding a new bin with conguration (b1; : : :; b ).
Algorithm 24.1 (; )-Precise
Input: I = ht1 : n1 ; : : :; t : n ; B i, where ti  B for all i
Output: a bin packing of I using minimum number of bins.
1. #H [i1; : : :; i ] = +1 for all 0  ij  nj , 1  j   ;
2. H [0; : : :; 0] = ; #H [0; : : :; 0] = 0;
3. for i1 = 0 to n1 do
4. for i2 = 0 to n2 do
.
.
.
5. for i = 0 to n do
6. for each bin configuration (b1; : : :; b)
addable to the subset [i1; : : :; i ] do
7. if #H [i1 + b1; : : :; i + b ] > 1 + #H [i1; : : :; i]
8. then
H [i1 + b1; : : :; i + b ] = H [i1; : : :; i]  (b1; : : :; b);
#H [i1 + b1; : : :; i + b ] = #H [i1; : : :; i ] + 1
Steps 7-8 can obviously be done in time O(n). Since bi  1= for all
i = 1; : : : , there are at most (1= ) addable bin congurations for each
subset [i1; : : :; i ]. Moreover, ni  n for all i = 1; : : :;  . Therefore, steps
7-8 can be executed at most n (1= ) times. We conclude that the running
119
time of the algorithm (;  )-Precise is bounded by O(n+1 (1= ) ), which
is a polynomial of n when  and  are xed.
The algorithm (;  )-Precise is not very satisfying. In particular, even
for a moderate constant  of dierent sizes, the factor n+1 in the complexity
makes the algorithm not practically useful. On the other hand, we will see
that our approximation algorithm for the general Bin Packing problem is
based on solving the (;  )-Bin Packing problem with a very large constant
 and a very small constant  . Therefore, we need, if possible, to improve
the above time complexity. In particular, we would like to see if there is an
algorithm that solves the (;  )-Bin Packing problem such that in the time
complexity of the algorithm, the exponent of n is independent of the values
of  and  .
Fix an input instance I = ht1 : n1 ; : : :; t : n ; B i for the (;  )-Bin
Packing problem. We say that a  -tuple (b1; : : :; b ) is a feasible bin con-
guration if bi  ni for all i and t1 b1 +


t b  B . Since ti  B for all i,
we get bi  1= for all i. Therefore, there are totally at most (1= ) feasible
bin congurations. Let all feasible bin congurations be
T1 = (b11; b12;


; b1)
T2 = (b21; b22;


; b2) (8)
..
.
Tq = (bq1; bq2;


; bq )
where q  (1= ) . Note that the above list of feasible bin P congurations can
be constructed in time independent of the number n = i=1 ni of objects
in the input instance I . Now each bin packing P of the input instance I
can be written as a q -tuple hx1; x2; : : :; xq i, where xj is the number of bins
of bin conguration Tj used in the packing P . Moreover, there is essentially
only one bin packing that corresponds to the q -tuple hx1; x2; : : :; xq i, if we
ignore the ordering of the bins used. An optimal packing corresponds to a
q-tuple hx1; x2; : : :; xq i with x1 +


xq minimized.
Conversely, in order to let a q -tuple hx1; x2; : : :; xq i to describe a real
pin packing, we need to make sure that the q -tuple uses exactly the input
objects given in I . For each feasible bin conguration Tj , there are bjh
objects of size th . Therefore, if xj bins are of bin conguration Tj , then for
the bin conguration Tj , the q -tuple assumes xj bjh objects of size th . Now
adding these over all bin congurations, we conclude that the total number

120
of objects of size th assumed by the q -tuple hx1 ; x2; : : :; xq i is
x1b1h + x2 b2h +


+ xq bqh
This should match the number nh of objects of size th in the input instance
I . This formulates the conditions into the following linear programming
problem.
min x1 + x2 +


+ xq
x1b11 + x2b21 +


+ xq bq1 = n1
x1b12 + x2b22 +


+ xq bq2 = n2 (9)
..
.
x1 b1 + x2 b2 +


+ xq bq = n
xi  0; for i = 1; : : :; q
Since all xi s must be integers, this is an integer linear programming problem.
It is easy to see that if a q -tuple hx1 ; : : :; xq i corresponds to a valid bin
packing of the input instance I , then the vector (x1; : : :; xq ) satises the
constraints in the system (9). Conversely, any vector (x1; : : :; xq ) satisfying
the constraints in the system (9) describes a valid bin packing for the input
instance I . Moreover, it is easy to see that if a vector (x1 ; : : :; xq ) satisfying
the constraints in the system (9) is given, the corresponding bin packing can
be constructed in linear time.
Therefore, to construct an optimal solution for the input instance I for
the (;  )-Bin Packing problem, we only need to construct an optimal
solution for the integer linear programming system (9). As we discussed
before, the Integer Linear Programming problem in general is NP-
hard. But here the nice thing is that both the number q of variablesPand the
number q + of constraints in the system (9) are independent of n = i=1 ni .
However, this does not immediately imply that the system can be solved in
time independent of n | the numbers ni appearing on the right side of the
system may be as large as n.
Anyway, the above system has at least suggested a polynomial time
algorithm for solving the problem: we know that an optimal solution must
satisfy x1 +


+ xq  n. Thus, 0  xi  n for all i = 1; : : :; q in an optimal
solution. Therefore, we could enumerate all vectors (x1; : : :; xq ) satisfying
0  xi  n and solve the system (9). Note that there are totally (n +1)q such
vectors and q is independent of n. However, since q has order (1= ) , this

121
enumerating algorithm gives a polynomial time algorithm whose complexity
is even worse than that of the algorithm (;  )-Precise.
Fortunately, Lenstra in 1983 has described an algorithm that solves the
system (9) in time h(q;  ), where h(q;  ) is a function depending only on q
and  . Since the algorithm involves complicated analysis on integer linear
programming, we omit the description of the algorithm.
We summarize the above discussion.
Algorithm 24.2 (; )-Precise2
Input: I = ht1 : n1 ; : : :; t : n ; B i, where ti  B for all i
Output: a bin packing of I using the minimum number of bins.

1. construct the list (8) of all feasible configurations

1 2 T ; T ; : : :; T
q;
2. solve the system (9) using Lenstra's algorithm;
3. return the solution 1 hx ; : : :; x i
q of step 2.
Algorithm (;  )-Precise2, as discussed above, runs in time h1 (q;  ) =
h2 (;  ), where h2 is a function depending only onP  and  . This may
seem a bit surprising since the algorithm packs n = i=1 ni objects in time
independent of n! This is really a matter of coding. Note that the input I =
ht1 : n1; : : :; t : n ; Bi of the algorithm (; )-Precise2 actually consists of
2 +1 integers, and the solution hx1; : : :; xq i given by the algorithm consists
of q = (1=) integers. P To convert the vector hx1; : : :; xqi into an actual
packing of the n = i=1 ni input objects, an extra step of time O(n) should
be added.
Theorem 24.1 The (; )-Bin Packing problem can be solved in time
O(n) + h(;  ), where h(;  ) is a function independent of n.

122
 CPSC-669 Computational Optimization
Lecture #25, October 25, 1995
Lecturer: Professor Jianer Chen
Scribe: Weijie Zhang
Revision: Jianer Chen
25 Approximating Bin Packing
In the last lecture, we have shown that the (;  )-Bin Packing problem can
be solved in time O(n) + h(;  ), where h(;  ) is a function independent of
n.
In today's lecture, we use the solution for the (;  )-Bin Packing prob-
lem to develop an approximation algorithm for the general Bin Packing
problem. Let us rst roughly describe the basic idea of the approximation
algorithm.
An input instance of the general Bin Packing problem may contain
objects of small size and objects of many dierent sizes. To convert an input
instance I = ht1 ; : : :; tn ; B i of the general Bin Packing problem to an input
instance of the (;  )-Bin Packing problem, we perform two preprocessing
steps:
1. ignore the objects of small size, i.e., the objects of size less than B ;
and
2. sort the rest of the objects by their sizes in decreasing order, then
partition the sorted list into  groups G1, : : :, G . For each group Gi ,
replace every object by the one with the largest size t0i in Gi .
After the preprocessing steps, we obtain an instance I 0 = ht01 : m; : : :; t0 :
m; Bi of the (; )-Bin Packing problem, where m  n= . Now we use the
algorithm we have described to construct a solution Y 0, which is a packing,
for the instance I 0. To obtain a solution Y to the original input instance
I of the Bin Packing problem, we rst replace each object in Y 0 by the
corresponding object in I , then add the objects in I that have size smaller
than B using greedy method.
The intuition is that an optimal solution to I 0 is an over-estimation of
the optimal solution to I (since each object in I is replaced by a larger
object in I 0, the number of bins used by an optimal packing of I 0 is at least
123
as large as the number of bins used by an optimal packing of I ); while an
optimal solution to I 00 = ht02 : m; : : :; t0 : m; B i is an under-estimation of
the optimal solution to I (I 00 can be regarded as an instance obtained by
replacing each object of I in the group Gi by a smaller object of size t0i+1
for 1  i   1 and deleting the objects in the last group G ). Since the
instance I 00 can also be obtained by deleting the m largest objects in I 0, an
optimal packing of I 0 uses at most m more bins than an optimal packing of
I 00 (the m bins are used to pack the m largest objects in I 0). Therefore, an
optimal packing of I 0 uses at most m more bins than an optimal packing
of I , with the objects of size less than B ignored. When the value  is
suciently large, the value m = n= is small so that an optimal solution to
I 0 will be a good approximation to the optimal solution to I with objects of
size less than B ignored.
Finally, after a good approximation of the optimal solution to the in-
stance I minus the small objects is obtained, we add the small objects to
this solution using greedy method. Since the small objects have small size,
the greedy method will not leave much room in each bin. Thus, the resulting
packing will be a good approximation for the input instance I of the general
Bin Packing problem.
We present the formal algorithm and formal analysis as follows.
Algorithm 25.1 ApprxBinPacking
Input: I = ht1 ; : : :; tn ; Bi
and 0 >
Output: a packing of the objects in I
1. sort t1 ; : : :; tn ; without loss of generality, let
t1  t2 


 tn
2. let h be the largest index such that th  B=2; let
I0 = ht1 ; t2; : : :; th ; Bi
3. let  d = e
= 4 2 , partition the objects in I0 into 
G ::: G
groups 1 , ,  , such that the group Gi consists
of the objects
t(m 1)i+1 ; t(m 1)i+2; : : :; tmi
where = m dh=e
(the last group  contains 0 G m m
objects).
4. construct an optimal solution 0 to the instance
Y
I 0 = ht1 : m; tm+1 : m; t2m+1 : m; : : :; t( 1)m+1 : m0 ; Bi
for the (=2;  )-Bin Packing problem;
5. replace each object in Y 0 of size tjm+1 by a proper

124
 object in the group G I
j +1 of 0 , for = 0 j1, ; : : :; 
Y
to construct a packing 0 for the instance 0 ; I
6. t
add the objects h+1 ; : : :; t I
n in to the packing 0 by Y
greedy method (i.e., no new bin will be used until no
used bin has enough space for the current object).
This results in a packing for the instance . I
According to Theorem 24.1 and note that  = d4=e, the (=2;  )-Bin
Packing problem can be solved in time O(n) + h(=2;  ) = O(n) + h0 (),
where h0 () is a function depending only on , we conclude that the algorithm
ApprxBinPacking runs in time O(n log n) + h0 (), if an O(n log n) time
sorting algorithm is used for step 1.
We discuss the approximation ratio for the algorithm ApprxBinPacking.
As before, we denote by Opt(I ) the optimal value, i.e., the number of bins
used by an optimal packing, of the input instance I of the Bin Packing
problem.
Lemma 25.1 Let I0 be the input instance constructed by step 2 of the al-
gorithm ApprxBinPacking. Then
Opt(I0)  Opt(I )

proof. This is because I0 is a subset of I so I takes at least as many bins

as I0 .
Lemma 25.2 Let I0 and I 0 be the input instances constructed by step 2 and
step 4 of the algorithm ApprxBinPacking, respectively. Then
Opt(I 0)  Opt(I0 )(1 + ) + 1

proof. Note that the instance I 0 is obtained from the instance I0 by

replacing each object in a group Gi by the largest object t(i 1)m+1 in the
group. Therefore, an optimal packing for the instance I 0 uses at least as
many bins as that used by an optimal packing for the instance I0 . This
gives
Opt(I0 )  Opt(I 0)
Now let
I 00 = htm+1 : m; t2m+1 : m; : : :; t( 1)m+1 : m0 ; Bi
125
I 00 can be regarded as an instance obtained from I0 by (1) replacing each
object in the group Gi by a smaller object tim+1 (recall that tim+1 is the
largest object in group Gi+1 ), for all i = 1; : : :;  2; (2) replacing m0 objects
in group G 1 by a smaller object t( 1)m+1 ; and (3) eliminating rest of the
objects in group G 1 and all objects in group G . Therefore, an optimal
packing for I0 uses at least as many bins as an optimal packing for I 00. This
gives
Opt(I 00)  Opt(I0)
Finally, the dierence between the instances I 0 and I 00 are m objects of size
t1 . Since an object can t into a bin, we must have
Opt(I 0)  Opt(I 00) + m
Combining all these we obtain
Opt(I0)  Opt(I 0)  Opt(I0 ) + m
This gives us
Opt(I 0)  Opt(I0 ) + m = Opt(I0) + dh=e
 Opt(I0) + h= + 1 = Opt(I0) + h2 =4 + 1 (10)
Now since each object of I0 has size at least B=2, each bin can hold at
most b2=c objects. Thus, the number of bins Opt(I0) used by an optimal
packing for the instance I0 is at least as large as h=2:
h=2  Opt(I0)
Use this in Equation (10), we get
Opt(I 0)  Opt(I0) + 
Opt(I0 )=2 + 1  Opt(I0 )(1 + ) + 1
The lemma is proved.
Lemma 25.3 The solution Y0 constructed by step 5 of ApprxBinPacking
is a packing for the instance I0. Moreover, the number of bins used by Y0 is
at most Opt(I0 )(1 + ) + 1.
proof. First note that the instances I0 and I 0 have the same number
of objects. The solution Y0 to I0 is obtained from the optimal solution Y 0
to the instance I 0 by replacing each of the m objects of size t(i 1)m+1 in I 0
126
by an object in group Gi of I0 . Since no object in group Gi has size larger
than t(i 1)m+1 , we actually replace objects in the bins in Y 0 by objects of
the same or smaller size. Therefore, no bin would get content more than B
in the packing Y0 . This shows that Y0 is a packing for the instance I0 .
Finally, since Y0 uses exactly the same number of bins as Y 0 and Y 0 is
an optimal packing for I 0 . By Lemma 25.2, the number of bins used by Y0 ,
i.e., the number of bins Opt(I 0) used by Y 0 , is at most Opt(I0)(1 + ) + 1.

Now we are ready for deriving our main theorem.

Theorem 25.4 For any input instance I = ht1; : : :; tn; Bi of the Bin Pack-
ing problem and for any 0 <   1, the algorithm ApprxBinPacking con-
structs a bin packing of I that uses at most Opt(I )(1 + ) + 1 bins.
proof. According to Lemma 25.3, the solution Y0 constructed by step
5 of the algorithm ApprxBinPacking is a packing for the instance I0 . Now
step 6 of the algorithm simply adds the objects in I I0 to Y0 using greedy
method. Therefore, the algorithm ApprxBinPacking constructs a packing
for the input instance I . Let Y be the packing constructed by the algorithm
ApprxBinPacking for I and let r be the number of bins used by Y . There
are two cases.
If in step 6 of the algorithm ApprxBinPacking, no new bin is introduced.
Then r equals the number of bins used by Y0 . According to Lemma 25.3
and Lemma 25.1, we get
r  Opt(I0)(1 + ) + 1  Opt(I )(1 + ) + 1
and the theorem is proved.
Thus, we assume that in step 6 of the algorithm ApprxBinPacking, new
bins are introduced. According to our greedy strategy, no new bin is intro-
duced unless no used bin has enough room for the current object. Since all
objects added by step 6 have size less than =2, we conclude that all of the
r bins in Y , except maybe one, have content larger than B(1 =2). This
gives us
t1 +


+ tn > B (1 =2)(r 1)
Therefore, an optimal packing of the instance I uses more than (1 =2)(r
1) bins. From
Opt(I ) > (1 =2)(r 1)
127
we derive
r < Opt(I )=(1 =2) + 1  Opt(I )(1 + ) + 1
The last inequality is because   1.
Therefore, in any case, the packing Y constructed by the algorithm
ApprxBinPacking for the input instance I of the Bin Packing problem
uses at most Opt(I )(1 + ) + 1 bins. The theorem is proved.
Note that the condition that  must be less than or equal to 1 loses no
generality. In particular, if we are interested in an approximation algorithm
for the Bin Packing problem with approximation ratio 1 +  with  > 1,
we simply use the First-Fit algorithm (Algorithm 23.1).
We conclude the lecture by the following theorem.
Theorem 25.5 The Bin Packing problem has an asymptotic polynomial
time approximation scheme.
proof. For any  > 0, let c = 2=. For each input instance I of the Bin
Packing problem, let the algorithm ApprxBinPacking construct in time
O(n log n) + h0(=2) a packing that uses at most r  Opt(I )(1 + =2) + 1
bins. Now for input instances I with Opt(I )  c = 2=, we have
r  1+  + 1  1+
Opt(I ) 2 Opt(I )
By the denition, the Bin Packing problem has an asymptotic polynomial
time approximation scheme.
The algorithm ApprxBinPacking runs in time O(n log n) + h0(), which
is not a polynomial of 1=. When  is small, the value h0 () can be huge.
Therefore, a further improvement on the algorithm ApprxBinPacking is
an algorithm of the similar approximation ratio but with the running time
bounded by a polynomial of n and 1=. This kind of algorithms is charac-
terized by the following denition.
Denition 25.1 An optimization problem Q = hI; S; f; opti has an asymp-
totic fully polynomial time approximation scheme (AFPTAS) if there is an
approximation algorithm A for Q such that for any  > 0, there is a constant
c such that for all input instances x 2 I with Opt(x)  c, the algorithm
A produces in time polynomial in both n and 1= a solution for x with
approximation ratio bounded by 1 + .
128
 The question whether the Bin Packing problem has an asymptotic
fully polynomial time approximation scheme was answered by Karmakar
and Karp, who use a similar approach that reduces the bin packing problem
to the linear programming problem. The algorithm uses some deep observa-
tions on the linear programming problem. We omit the detailed description
here. Instead, we state the result directly.
Theorem 25.6 (Karmakar and Karp) There is an approximation algorithm
A for the Bin Packing problem such that for any  > 0, the algorithm A
produces in time polynomial in n and 1= a packing in which the number of
bins used is bounded by
Opt(x)(1 + ) + 1=2 + 3
Corollary 25.7 The Bin Packing problem has an asymptotic fully poly-
nomial time approximation scheme.
proof. For any  > 0, let c = (8 + 62 )=3 . For each input instance I
of the Bin Packing problem, let the Karmakar-Karp algorithm construct
a packing that uses at most
r  Opt(I )(1 + =2) + (2=)2 + 3
bins. Now for input instances I with
Opt(I )  c = (8 + 62 )=3
we have
r  1 +  + (2=)2 + 3  1 + 
Opt(I ) 2 Opt(I )
Moreover, the algorithm runs in time polynomial in n and 2=, which is also
in polynomial in n and 1=.

129
 CPSC-669 Computational Optimization
Lecture #26, October 27, 1995
Lecturer: Professor Jianer Chen
Scribe: Li Shao
Revision: Jianer Chen
26 Multi-processor scheduling
In the next few lectures, we study how the techniques developed for the Bin
Packing problem can be used to develop a polynomial time approximation
scheme for the Multi-Processor Scheduling problem.
Recall that the Multi-Processor Scheduling problem is dened as
follows.
Multi-Processor Scheduling
Input: ht1 ; t2; : : :; tn ; mi, all integers, where ti is the processing
time for the ith job
Output: a scheduling of the n jobs on m identical processors
such that the parallel nish time is minimized
We point out a few properties for the Multi-Processor Scheduling
problem:
1. Even if we x the number m of processors to be any constant larger
than 1, the problem is still NP-hard (Theorem 13.3);
2. If the number m of processors is a xed constant, then the problem
has a fully polynomial time approximation scheme (Corollary 16.2).
3. If m is not xed, the problem is strongly NP-hard and has no fully
polynomial time approximation scheme unless P = NP (see Lecture
Notes #18).
Therefore, the best we can expect for the Multi-Processor Schedul-
ing problem is a (non-fully) polynomial time approximation scheme.
The Multi-Processor Scheduling problem can also be regarded as
a variation of the Bin Packing problem in which we are given n objects of
sizes t1 , : : :, tn , respectively, and the number m of bins, and we are asked

130
to pack the objects into the m bins such that the bin size is minimized.
Therefore, there are two parameters: the number of bins and the bin size.
Each of the Multi-Processor Scheduling problem and the Bin Pack-
ing problem xes one parameter and optimizes the other parameter. In this
sense, the Multi-Processor Scheduling problem is \dual" to the Bin
Packing problem. Therefore, it is not very surprising that the techniques
developed for approximation algorithms for the Bin Packing problem can
be useful in deriving approximation algorithms for the Multi-Processor
Scheduling problem.
Consider the following problem, where for an input instance I of the Bin
Packing problem, we use Opt(I ) to denote the optimal value of I , i.e., the
number of bins used by an optimal packing of the instance I .
(1 + )-Bin Packing
Input: I = ht1 ; t2; : : :; tn ; B i, all integers
Output: a packing of the n objects into at most Opt(I ) bins
such that the content of each bin is at most (1 + )B
We rst show that the (1 + )-Bin Packing problem can be solved
in polynomial time for a xed constant  > 0. Then we show how this
solution can be used to derive a polynomial time approximation scheme for
the Multi-Processor Scheduling problem.
The idea for solving the (1 + )-Bin Packing problem is very similar to
the one for the approximation algorithm ApprxBinPacking for the general
Bin Packing problem. We rst perform two preprocessing steps:
1. ignore the objects of size less than B ; and
2. partition the rest of the objects into  groups G1 , : : :, G so that the
objects in each group have a very small dierence in size. For each
group Gi , replace every object by the one with the smallest size in Gi .
The preprocessing steps give us an instance I 0 of the (;  )-Bin Packing
problem, for which an optimal solution can be constructed in polynomial
time. Note that the optimal solution for I 0 is an under-estimation of the
optimal solution for I and thus it uses no more than Opt(I ) bins. Then we
restore the object sizes and add the small objects by greedy method to get
a packing for the instance I . Since the dierence in sizes of the objects in
each group is very small, the restoring of object sizes will not increase the

131
content for each bin very much. Similarly, adding small objects using greedy
method will not induce much error.
The formal algorithm is given as follows.
Algorithm 26.1 VaryBinPacking
Input: I = ht1 ; : : :; tn ; Bi
and 0 >
Output: a packing of the objects of I into bins of size
(1 + )B
1. sort t1 ; : : :; tn ; without loss of generality, let
t1  t2 


 tn
2. let h be the largest index such that th > B ; let
I0 = ht1 ; t2; : : :; th ; Bi
3. let  d = e
= 1 2 , divide the line segment (B; B ] into 
subsegments of equal length
(l1; h1]; (l2; h2]; : : :; (l ; h ]
h
where i = i+1 and i l i=( h
) ; l B B =
4. partition the objects in 0 into I
groups 1 , , , G ::: G
G
such that an object is in group i if and only if its
size is in the range ( i i]; let 0i be the size of the
l ;h t
smallest object in group i (if i is empty, letG G
0 = i ), and let i be the number of objects in i;
t l m G
i
5. construct an optimal solution 0 to the instance
Y
I 0 = ht01 : m1 ; t02 : m2; : : :; t0 : m ; Bi
for the (;  )-Bin Packing problem;
6. replace each object in Y 0 of size t0j by a proper object
in the group Gj of I0 , for j = 1; : : :;  , to construct a
packing Y0 for the instance I0 ;
7. add the objects th+1 ; : : :; tn in I to the packing Y0 by
greedy method (i.e., no new bin will be used until
adding the current object to any used bins would
exceed B
the size (1 + ) ). This results in a packing
Y for the instance . I
According to Theorem 24.1 and note that  = d1=2 e, the (;  )-Bin
Packing problem can be solved in time O(n)+ h0 (), where h0 () is a func-
tion depending only on . We conclude that the algorithm VaryBinPacking
runs in time O(n log n) + h0 (), if an O(n log n) time sorting algorithm is
used for step 1.

132
 As before, we denote by Opt(I ) the optimal value, i.e., the number of
bins used by an optimal packing, of the input instance I of the general Bin
Packing problem.
Lemma 26.1 The packing Y 0 for the instance I 0 constructed by step 5 of
the algorithm VaryBinPacking uses no more than Opt(I ) bins.
proof. The instance I0 is a subset of the instance I . Thus, Opt(I0) 
Opt(I ). The instance I 0 is obtained from I0 by replacing each object in I0 by
a smaller object. Thus, Opt(I 0)  Opt(I0 )  Opt(I ). Since Y 0 is an optimal
packing for I 0 , Y 0 uses Opt(I 0)  Opt(I ) bins.
Lemma 26.2 In the packing Y0 constructed by step 6 of VaryBinPacking,
no bin has content larger than (1 + )B , and Y0 uses no more than Opt(I )
bins.
proof. According to step 6 of the algorithm VaryBinPacking, the number
of bins used by Y0 is the same as that used by Y 0. By Lemma 26.1, the
packing Y0 uses no more than Opt(I ) bins.
Each object of size t0i in I 0 corresponds to an object in group Gi in I0 .
The packing Y0 for I0 is obtained from the packing Y 0 by restoring each
object of I 0 to the corresponding object in I0 . Since t0i is the size of the
smallest object in Gi and no object in Gi has size larger than
t0i + (hi li) = t0i + (B B)=
the size increase for each object from Y 0 to Y0 is bounded by (B B )= .
Moreover, since all objects in I 0 have size at least B , and the packing
Y 0 has bin size B, each bin in the packing Y 0 holds at most b1=c objects.
Therefore, the size increase for each bin from Y 0 to Y0 is bounded by
((B B )= )
b1=c = ((B B )=d1=2 e)
b1=c  (B=(1=2))
(1=) = B
Since the content of each bin of the packing Y 0 is at most B , we conclude
that the content of each bin of the packing Y0 is at most (1 + )B .
Lemma 26.3 The packing Y constructed by step 7 of VaryBinPacking uses
no more than Opt(I ) bins, and each bin of Y has content at most (1 + )B .
proof. By Lemma 26.2, each bin of the packing Y0 has content at most

133
(1 + )B . The packing Y is obtained from Y0 by adding the objects of size
bounded by B using greedy method. That is, suppose we want to add an
object of size not larger than B and there is a used bin whose content will
not exceed (1 + )B after adding the object to the bin, then we add the
object to the bin. A new bin is introduced only if no used bin can have the
object added without exceeding the content (1 + )B . The greedy method
ensures that the content of each bin in Y is bounded by (1 + )B . Note
that since all added objects have size bounded by B , when a new bin is
introduced, all used bins have content larger than B .
If no new bin was introduced in the process of adding small objects in
step 7, then the number of bins used by the packing Y is the same as the
number of bins used by the packing Y0 . By Lemma 26.2, in this case the
packing Y uses no more than Opt(I ) bins.
Now suppose that new bins were introduced in the process of adding
small objects in step 7. Let r be the number of bins used by the packing
Y . By the above remark, at least r 1 bins in the packing Y have content
larger than B . Therefore, we have
t1 +


+ tn > B (r 1)
This shows that we need more than r 1 bins of size B to pack the objects in
I in any packing. Consequently, the value Opt(I ) is at least (r 1) + 1 = r.
That is, the packing Y uses no more than Opt(I ) bins.
We conclude this lecture with the following theorem.
Theorem 26.4 Given an instance I = ht1; : : :; tn; Bi for the Bin Packing
problem and a constant  > 0, The algorithm VaryBinPacking constructs
in time O(n log n) + h0 () a packing for I that uses no more than Opt(I )
bins and the content of each bin is bounded by (1 + )B , where Opt(I ) is
the number of bins used by an optimal packing of I using bins of size B and
h0 () is a function depending only on .
Corollary 26.5 The (1 + )-Bin Packing problem can be solved in poly-
nomial time for a xed constant .

134
 CPSC-669 Computational Optimization
Lecture #27, October 30, 1995
Lecturer: Professor Jianer Chen
Scribe: Li Shao
Revision: Jianer Chen
27 Approximating multi-processor scheduling
In the last lecture, we developed an algorithm VaryBinPacking that, given
an input instance I = ht1 ; : : :; tn ; B i of the Bin Packing problem and a
constant  > 0, constructs in time O(n log n) + h0 () a packing using at
most Opt(I ) bins such that the content of each bin is bounded by (1 + )B ,
where h0 () is a function depending only on .
We use this algorithm to develop a polynomial time approximation scheme
for the Multi-Processor Scheduling problem. We rst re-formulate the
Multi-Processor Scheduling problem in the language of bin packing.
Multi-Processor Scheduling (Bin Packing version)
Input: ht1 ; t2; : : :; tn ; mi, all integers, where ti is the size of the
ith object
Output: a packing of the n objects into m bins of size B with
B minimized
We use the idea of binary search to nd the optimal bin size B . In
general, suppose that we try bin size B , and nd out that the input instance
ht1; : : :; tn; Bi for the Bin Packing problem needs more than m bins in its
optimal packing, then the tried bin size B is too small. So we will try a
larger bin size. On the other hand, if the instance ht1 ; : : :; tn ; B i needs no
more than m bins, then we may want to try a smaller bin size because we
are minimizing the bin size. Note that the algorithm VaryBinPacking can
be used to estimate the number of bins used by an optimal packing of the
instance ht1 ; : : :; tn ; B i.
We rst discuss the initial bounds for the bin size in the binary search.
Fix an input instance ht1 ; : : :; tn ; mi for the Multi-Processor Schedul-
ing problem. Let
X
n
Avg = maxf ti =m; t1; t2; : : :; tng
i=1

135
Lemma 27.1 The minimum bin size of the input instance ht1; : : :; tn; mi
for the Multi-Processor Scheduling problem is at least Avg.
proof. Since
Pn t =m is the average content of the m bins for packing
i=1 i
the n objects of size t1 , : : :, tn , any packingPof the n objects into the m bins
has at least one bin with Pcontent at least ni=1 ti =m. That is, the bin size
of the packing is at least n t =m.
i=1 i
Moreover, the bin size of the packing should also be at least as large as
any ti since every object has to be packed into a bin in the packing.
This shows that for any packing of the n objects of size t1 , : : :, tn into
the m bins, the bin size is at least Avg. The lemma is proved.
This gives a lower bound on the bin size for the input instance I of
the Multi-Processor Scheduling problem. We also have the following
upper bound.
Lemma 27.2 The minimum bin size of the input instance ht1; : : :; tn; mi
for the Multi-Processor Scheduling problem is bounded by 2
Avg.
proof. Suppose that the lemma is false. Let r be the minimum bin size
for packing I = ht1; : : :; tn ; mi into m bins, and r > 2
Avg.
Let Y be a packing of I into m bins such that the bin size of Y is r.
Furthermore, we suppose that Y is the packing in which the least number
of bins have content r. Let B1 , B2 , : : :, Bm be the bins used by Y , where
the bin B1 has content r > 2
Avg. Then at least one of the bins B2 , : : :,
Bm has content less than Avg | otherwise, the sumPof total contents of the
bins B1 , B2 , : : :, Bm would be larger than mAvg  ni=1 ti . Without loss of
generality, suppose that the bin B2 has content less than Avg. Now remove
any object ti in the bin B1 and add ti to the bin B2 . We have
1. the content of the bin B1 in the new packing is less than r;
2. the content of the bin B2 in the new packing is less than
Avg + ti  2
Avg < r
3. the contents of the other bins are unchanged.
Thus, in the new packing, the number of bins that have content r is one less
than the number of bins of content r in the packing Y . This contradicts our
assumption that Y has the least number of bins of content r.
This contradiction proves the lemma.
136
 Therefore, the minimum bin size for packing the instance I into m bins
is in the range [Avg; 2Avg]. We apply binary search on this range to nd
an approximation for the optimal solution of I for the Multi-Processor
Scheduling problem.
Algorithm 27.1 ApprxMPS
Input: I = ht1; : : :; tn ; mi,
all integers, and 0 >
Output: a scheduling of the n
jobs of processing time t1 ,
t
2,::: t
, n on m
identical processors.
P
Avg = maxf n
1. i=1 ti =m; t1; t2; : : :; tn g;
2. lower = bAvgc; upper = d2
Avge;
3. while upper lower > 
Avg=4 do
B = b(lower + upper)=2c;
call the algorithm VaryBinPacking on the input
ht ; : : :; t Bi
1 n; and =
4; suppose that the algorithm
uses r
bins on the input;
if r > m
then lower = B
else upper = B;
4. let B = upper(1 + 4) ;
b = c
5. call the algorithm VaryBinPacking on the input
1 n ;  and 4 to construct a scheduling
ht ; : : :; t B i =
of . I
We rst study the complexity of the above algorithm ApprxMPS. The
complexity of the algorithm is dominated by step 3. We start with
upper lower = 2
Avg Avg = Avg
Since we are using binary search, each execution of the body of the while
loop will half the dierence (upper lower). Therefore, after O(log(1=))
executions of the body of the while loop in step 3, we must have
upper lower  
Avg=4
That is, the body of the while loop is executed at most O(log(1=)) times.
In each execution of the body of the while loop in step 3, we call the
algorithm VaryBinPacking on input ht1 ; : : :; tn ; B i and =4, which takes
time O(n log n) + h0 (=4) = O(n log n) + h1 (), where h1 () is a function

137
depending only on . Therefore, the running time of the algorithm ApprxMPS
is bounded by
O(log(1=))(O(n log n) + h1 ()) = O(n log n log(1=)) + h2()
where h2 () is a function depending only on .
Theorem 27.3 The running time of the algorithm ApprxMPS on input in-
stance I = ht1 ; : : :; tn ; mi and  > 0 is bounded by O(n log n log(1=))+ h2().
In particular, for a xed constant  > 0, the algorithm ApprxMPS runs in
polynomial time.
We will present the analysis for the approximation ratio for the algorithm
ApprxMPS in the next lecture.

138
 CPSC-669 Computational Optimization
Lecture #28, November 1, 1995
Lecturer: Professor Jianer Chen
Scribe: Li Shao
Revision: Jianer Chen
28 More on multi-processor scheduling
In the last lecture, we presented the following algorithm for the Multi-
Processor Scheduling problem.
Algorithm 28.1 ApprxMPS
Input: I = ht1; : : :; tn ; mi,
all integers, and 0 >
Output: a scheduling of the n
jobs of processing time t1 ,
t
2,::: t
, n on m identical processors.
P t =m; t ; t ; : : :; t g;
Avg = maxf n
1. i=1 i 1 2 n
2. lower = bAvgc; upper = d2
Avge;
3. while upper lower > 
Avg=4 do
B = b(lower + upper)=2c;
call the algorithm VaryBinPacking on the input
ht ; : : :; t Bi
1 n; and =
4; suppose that the algorithm
uses r
bins on the input;
if r > m
then lower = B
else upper = B;
4. let B = upper(1 + 4) ;
b = c
5. call the algorithm VaryBinPacking on the input
1 n ;  and 4 to construct a scheduling
ht ; : : :; t B i =
of . I
we also showed that the algorithm runs in time O(n log n log(1=)) + h2 (),
where h2 () is a function depending only on . Now we discuss the approx-
imation ratio of the algorithm.
Fix an input instance I = ht1 ; : : :; tn ; mi for the Multi-Processor
Scheduling problem. Let Opt(I ) be the optimal solution, i.e., the parallel
nish time of an optimal scheduling, of the instance I .

139
Lemma 28.1 In the whole execution of the algorithm ApprxMPS, we always
have
lower  Opt(I )  upper(1 + =4)

proof. Initially, lower = bAvgc and upper = d2
Avge. By Lemmas 27.1

and 27.2, we have lower  Opt(I )  upper(1 + =4).
Now for each execution of the while loop in step 3, we start with a bin
size B and call the algorithm VaryBinPacking on input ht1 ; : : :; tn ; B i and
=4, which uses r bins.
If r > m, by the algorithm VaryBinPacking, the minimum number of
bins used by a packing to pack the objects into bins of size B is at least as
large as r. Therefore, if the bin size is B , then we need more then m bins
to pack the objects t1 , : : :, tn . Thus, in order to pack the objects t1 , : : :, tn
into m bins, the bin size B is too small. That is, Opt(I ) > B . Since in this
case we set lower = B , the relation lower  Opt(I )  upper(1 + =4) still
holds.
If r  m, then the objects t1 , : : :, tn can be packed in r bins of size
(1 + =4)B . Certainly, the objects can also be packed in m bins of size
(1 + =4)B . This gives Opt(I )  (1 + =4)B . Thus, setting upper = B still
keeps the relation lower  Opt(I )  upper(1 + =4).
This proves the lemma.
Now we are ready to show that the algorithm ApprxMPS is a polyno-
mial time approximation scheme for the Multi-Processor Scheduling
problem.
Theorem 28.2 On any input instance I = ht1; : : :; tn; mi for the Multi-
Processor Scheduling problem and for any , 0 <   1, the algorithm
ApprxMPS constructs in time O(n log n log(1=)) + h2 () a scheduling of the
n jobs on the m processors with approximation ratio 1 + , where h2 () is a
function depending only on .
proof. The time complexity of the algorithm ApprxMPS is given by
Theorem 27.3.
By Lemma 28.1, the relation
lower  Opt(I )  upper(1 + =4)
always holds. In particular, at step 4 of the algorithm, we have
Opt(I )  upper(1 + =4)
140
Since Opt(I ) is an integer, we should also have
Opt(I )  bupper(1 + =4)c = B
Therefore, the bin of size B  is at least as large as the bin of size Opt(I ).
Since the objects t1 , : : :, tn can be packed into m bins of size Opt(I ), we
conclude that the objects t1 , : : :, tn can also be packed into m bins of size B  .
By the property of the algorithm VaryBinPacking, on input instance I =
ht1; : : :; tn; Bi and =4, the algorithm VaryBinPacking packs the objects
t1 , : : :, tn into at most m bins, with each bin of content at most B  (1+ =4).
Therefore, the packing is a scheduling of the n jobs on the m processors.
Now let us consider the content bound B  (1 + =4) for the bins in the
packing constructed by the algorithm VaryBinPacking. At step 4, we have
upper lower  
Avg=4
Since lower = Avg initially, and lower is never decreased, we have
upper  lower + 
Avg=4  lower + 
lower=4 = lower(1 + =4)
By Lemma 28.1, we always have
lower  Opt(I )  upper(1 + =4)
Thus
upper(1 + =4)  lower(1 + =4)2  Opt(I )(1 + =4)2
Therefore, the content bound B  (1 + =4) is bounded by
B (1 + =4) = bupper(1 + =4)c(1 + =4)
 upper(1 + =4)2  Opt(I )(1 + =4)3
Now Opt(I )(1+=4)3  Opt(I )(1+) for   1. Recall that in the scheduling,
the number m of bins corresponds to the number of processors, and the
maximum bin content B  (1 + =4) corresponds to the parallel nish time.
In conclusion, the scheduling of the n jobs on the m processors constructed
by the algorithm ApprxMPS has parallel nish time bounded by Opt(I )(1+ ).
In other words, the algorithm ApprxMPS has approximation ratio 1 + .
Corollary 28.3 The Multi-Processor Scheduling problem has a poly-
nomial time approximation scheme.

141
 Again, the condition   1 is not crucial. In particular, we will see below
that if  > 1, a much simpler approximation algorithm for the Multi-
Processor Scheduling problem can be designed to have approximation
ratio bounded by 1 + .
Algorithm 28.2 SimpleMPS
Input: I = ht1; : : :; tn ; mi,
all integers
Output: a scheduling of the n
jobs of processing time t1 ,
t
2 , ,::: t
n on m
identical processors

for i = 1 to n do
assign ti to the processor with the lightest load;

Using a data structure such as a 2-3 tree to organize the m processors

using their loads as the keys, we can always nd the lightest loaded pro-
cessor, update its load, and re-insert it back to the data structure in time
O(log m). With this implementation, the algorithm SimpleMPS runs in time
O(n log m).
Now we study the approximation ratio of the algorithm SimpleMPS.
Theorem 28.4 Algorithm SimpleMPS for the Multi-Processor Schedul-
ing problem has approximation ratio bounded by 2.
proof. Let I = ht1 ; : : :; tn ; mi be an input instance to the Multi-
Processor Scheduling problem. Suppose that the algorithm SimpleMPS
constructs a scheduling S for I with parallel nish time T . Let P1 be a
processor that has the execution time T assigned by the scheduling S .
If the processor P1 is assigned only one job, then the job has processing
time T , and any scheduling on I has parallel nish time at least T . In this
case, the scheduling S is a optimal scheduling with approximation ratio 1.
So suppose that the processor P1 is assigned at least two jobs. Let t0
be the last job assigned to the processor P1 . We have T t0 > 0. By our
strategy, at the time the job t0 is about to be assigned to the processor P1 ,
all processors have load at least T t0 . This gives:
X
n
ti  m(T t0) + t0 = mT (m 1)t0
i=1
This gives
Pn t + (m 1)t
T  i=1 i m 0

142
 Pn t m 1
i=1 i +
=
Pnm t m t0
 i=1 i
m + t0
P
Now since the optimal value Opt(I ) is at least as large as ( ni=1 ti )=m, and
at least as large as t0 , we conclude that
T  2
Opt(I )
Consequently, the approximation algorithm SimpleMPS has approximation
ratio bounded by 2.
There are certainly many possible ways to improve the performance of
the algorithm SimpleMPS. For example, it seems that if we sort the jobs rst
so that the larger jobs will be assigned rst, then we may get an improvement
on the approximation ratio. In fact, it can be shown that such a modication
makes the algorithm have an approximation ratio of 4=3. Students are
encouraged to think of other possible improvements.

143
 CPSC-669 Computational Optimization
Lecture #29, November 3, 1995
Lecturer: Professor Jianer Chen
Scribe: Shijin Lu
Revision: Jianer Chen
29 Approximability with a constant ratio
So far we have seen many optimization problems that can be approximated
in polynomial time to approximation ratio 1 + , for any given constant .
These problems are classied into the following two classes.
Denition 29.1 An optimization problem is in the class FPTAS if it has
a fully polynomial time approximation scheme. An optimization problem is
in the class PTAS if has a polynomial time approximation scheme.
Obviously, FPTAS is a subclass of PTAS.
On the other hand, there are many other optimization problems that
do not seem to have such nice approximability. There is a large class of
optimization problems of practical importance, which do not seem to have
polynomial time approximation schemes. The rest of this course will be
centered on the study of these optimization problems.
Let us rst consider the Vertex Cover problem. Given a graph G =
(V; E ), we say that a subset V0 of V is a vertex cover of the graph G if every
edge of the graph G has at least one endpoint in V0.
Vertex Cover
Input: an undirected graph G = (V; E )
Output: a vertex cover V0 of minimum cardinality
The Vertex Cover problem has applications in computer networks,
VLSI design, and circuit testing. For example, in computer network, we are
given a network, which can be regarded as a graph, and we are asked to
pick a set of nodes in the network so that all connections of the network are
monitored by the nodes in the set. To economize the resources, we expect
to have as few nodes as possible in the set. This is exactly the Vertex
Cover problem.
We have a very ecient and simple approximation algorithm for the
Vertex Cover problem. The algorithm is given below.

144
Algorithm 29.1 ApprxVC
Input: an undirected graph G = (V; E )
Output: a vertex cover of G
1. Let V0 = ;
2. for each edge e of G do
if e has no ends in V0
then add both ends of e to V0
From the ApprxVC algorithm, we can easily get two observations.
Observation 29.2 The set V0 constructed by the algorithm ApprxVC is a
vertex cover of the graph G.
As we can see, the algorithm makes sure that all edges of the graph G
are covered by the set V0 .
Observation 29.3 The algorithm ApprxVC actually constructs a maximal
matching for the graph G.
When the algorithm ApprxVC includes two endpoints u and v of an edge
e in the set V0, we can regard that the algorithm matches the two endpoints
u and v by the edge e. By the algorithm, if the endpoints u and v are
matched by e, no other edge incident on either u or v would be used for
matching. That is, the set
E0 = fe j e is picked by ApprxVC for matching its two endsg
is a matching in G. Moreover, the matching is maximal because every edge
has at least one end in V0 .
Theorem 29.1 The algorithm ApprxVC is an approximation algorithm for
the Vertex Cover problem and has approximation ratio 2.
proof. By Observation 29.2, the algorithm ApprxVC always constructs a
vertex cover for the input graph G.
By Observation 29.3, a maximal matching E0 is constructed by the al-
gorithm ApprxVC. Let
E0 = fe j e is picked by ApprxVC for matching its two endsg

145
and let C be any minimum vertex cover. Then every edge in E0 should be
covered by C , i.e., each edge in E0 should have at least one end in C . Since
no two edges in E0 share a common end, we should have
jC j  jE0j
Since each edge in E0 has two ends in V0 and no two edges in E0 share a
common end, we have
2jE0j = jV0j
In conclusion
Opt(G) = jC j  jV0j=2
This gives the approximation ratio
jV0j=Opt(G)  2
and the theorem is proved.
The algorithm ApprxVC looks very simple. However, it gives the best
approximation ratio known for the Vertex Cover problem. Actually, it is
an outstanding open problem whether the Vertex Cover problem has a
polynomial time approximation algorithm with approximation ratio r < 2,
for a xed constant r > 0.
I assign the following as one of the project problems.
Project problem: Improve the approximation ratio 2 for the Vertex
Cover problem on graph classes with some reasonable restrictions.
There are many optimization problems like the Vertex Cover problem
that have polynomial time approximation algorithms with approximation
ratio bounded by a xed constant c (c = 2 for the Vertex Cover problem).
On the other hand, for many of them, it is unknown whether the constant
c can be arbitrarily close to 1, i.e., whether the problems have polynomial
time approximation schemes. We discuss another example as follows.
Let X , Y , and Z be three nite sets. Given a subset S  X  Y  Z ,
a matching M in S is a subset of S such that no two triples in M have
the same coordinate at any dimension. The 3-Dimensional Matching
problem is dened as follows.
3-D Matching
Input: a set S  X  Y  Z of triples
Output: a matching M in S with jM j maximized

146
 The 3-D Matching problem is a generalization of the classical \mar-
riage problem": Given n unmarried men and m unmarried women, along
with a list of all male-female pairs who would be willing to marry one an-
other, nd the largest number of pairs so that polygamy is avoided and
every paired person receives an acceptable spouse. Analogously, in the 3-D
Matching problem, the sets X , Y , and Z correspond to three sexes, and
each triple in S corresponds to a 3-way marriage that would be acceptable
to all three participants.
Remark 29.4 The 2-D Matching problem can be similarly dened: given
a set S  X  Y of pairs, nd a maximum subset M of S such that no two
pairs in M agree in any coordinate. The 2-D Matching problem is the
standard graph matching problem. In fact, the sets X and Y can be regarded
as the vertices of a graph G, and each pair in the set S corresponds to an
edge in the graph G. Now a matching M in S is simply a subset of edges in
which no two edges share a common end. That is, a matching in S is a graph
matching in the corresponding graph G. As we have studied in Lectures 8-
10, the graph matching problem, i.e., the 2-D Matching problem can be
solved in polynomial time.
Remark 29.5 The 3-D Matching problem is NP-hard. This is from the
fact that the decision version of the 3-D Matching problem is NP-complete
(see Garey and Johnson's book) and can be reduced to the optimization
version of the 3-D Matching problem. In fact, the decision version of the
3-D Matching problem is listed by Garey and Johnson as one of the six
basic NP-complete problems.
We present two polynomial time approximation algorithms for the 3-D
Matching problem.
Let S  X  Y  Z be a set of triples and let M be a matching of S . We
say that a triple (x; y; z ) in S M does not contradict the matching M if no
triple in M has x as its rst coordinate, or has y as its second coordinate,
or has z as its third coordinate. In other words, (x; y; z ) does not contradict
the matching M if M [ f(x; y; z )g is still a matching.
Algorithm 29.2 Apprx3D-First
Input: a set S XY Z of triples
Output: a matching M in S
1. let M = .
147
 2. for each triple (x; y; z) in S do
if (x; y; z) does not contradict M
then add (x; y; z) to M .
It is easy to verify that the algorithm Apprx3D-First runs in polynomial
time. In fact, if we use three arrays for the symbols in X , Y , and Z , and
mark the symbols as \in M " or \not in M ", then in constant time we can
decide whether a triple (x; y; z ) contradicts the matching M . With these
data structures, the algorithm Apprx3D-First runs in linear time.
Theorem 29.2 The algorithm Apprx3D-First constructs a matching in
the set S and has approximation ratio 3.
proof. From the algorithm Apprx3D-First, it is clear that the set M
constructed is a matching in the given set S .
Let Mmax be a maximum matching in S and let (x; y; z ) be a triple in
Mmax. By the algorithm Apprx3D-First, the triple (x; y; z ) contradicts the
matching M (otherwise, it would have been added to M by the algorithm).
Therefore, either x is the rst coordinate of a triple in M , or y is the second
coordinate of a triple in M , or z is the third coordinate of a triple in M .
Therefore, the total number of symbols appearing in the matching M (in
either the rst dimension, or the second dimension, or the third dimension)
is at least jMmaxj. Since each triple in M uses exactly three symbols, we
conclude that the number of triples in the matching M is at least jMmaxj=3.
That is,
Opt(S )=jM j = jMmaxj=jM j  3
The theorem is proved.

148
 CPSC-669 Computational Optimization
Lecture #30, November 6, 1995
Lecturer: Professor Jianer Chen
Scribe: Shijin Lu
Revision: Jianer Chen
30 3-dimensional matching
We continue our discussion on the 3-D Matching problem.
Let S  X  Y  Z be a set of triples. Without loss of generality, we
assume that the symbol sets X , Y , and Z are all pairwise disjoint. Therefore,
it makes no ambiguity to say that a triple t contains a symbol w in X [ Y [ Z .
Recall that a matching M in S is a subset of S in which no two triples agree
in any coordinate. We say that a symbol w 2 X [ Y [ Z is in the matching
M if a triple in M contains the symbol w. A triple t in S M contradicts
the matching M if a symbol in t is also in the matching M . We say that a
matching M in S is maximal if every triple in S M contradicts M .
Before we present another approximation algorithm for the 3-D Match-
ing problem, we diverge to a related problem.
k-Triple Matching
Given a set S  X  Y  Z of n triples and an integer k > 0, nd
a matching in S with k triples or report that no such a matching
exists in S .
It is clear that the k-Triple Matching problem can be solved in time
O(nk ) if we pick every k triples in S and check whether they make a match-
ing. However, the algorithm will be very time-consuming even for a small
value of k. We would like to have a better algorithm for the problem. In
particular, we would like to have an algorithm for the problem such that in
the time complexity of the algorithm, the exponent of n is independent of
the value k.
We present an algorithm solving the k-Triple Matching problem as
follows. The algorithm is rst given as a nondeterministic algorithm, i.e.,
an algorithm that can \guess" a desired object in a set without exhaustively
searching the set. Then we show how the nondeterministic algorithm can
be converted into a deterministic one.

149
 We rst suppose that a maximum matching in the set S contains at
least k triples. Fix a matching M0 = ft1 ; t2; : : :; tk g of k triples in S (the
matching M0 is unknown to our algorithm).
Let M1 be a maximal matching in S . M1 can be found in time O(n)
by, say, the algorithm Apprx3D-First given in the last lecture. If jM1 j  k,
then we are done | any k triples in M1 make a matching of k triples in S .
Thus, we assume jM1j < k.
Let ti be any triple in the matching M0. If ti is in M1, then certainly
the symbols in ti are also in M1 . If ti is not in M1 , then ti contradicts M1
because M1 is maximal. Thus, in any case, for each triple ti in M0 at least
one symbol in ti is in the matching M1 .
Thus, our algorithm guesses k symbols a1, : : :, ak in M1 such that ai is
a symbol in the triple ti , i = 1; : : :; k. This gives us a \pseudo-matching"
M2 = ft(2) (2) (2)
1 ; t2 ; : : :; tk g
where t(2)
i is the triple ti in the matching M0 with the symbol ai present
and the other two symbols replaced by a special symbol `', for i = 1 : : :; k.
This gives us the initial pseudo-matching. Note that the pseudo-matching
M2 can be constructed from the matching M1 in time O(k) if the guessed
symbols a1, : : :, ak are given.
Inductively, suppose that we have obtained a pseudo-matching
Mj = ft(1j) ; t(2j); : : :; t(kj)g
where t(i j ) is the triple ti in the matching M0 with at least one symbol present
and the other symbols replaced by the symbol `', for i = 1 : : :; k. We say
that a triple t in S is consistent with a triple t(i j ) if t and t(i j ) agree in all
coordinates except those on which t(i j ) has the symbol `'.
Now we try to ll the missing symbols in the pseudo-matching Mj using
a greedy algorithm. Formally, we start with M 0 =  then scan the triples
in S . We add a triple t in S to M 0 if (1) t is consistent with a triple t(i j ) in
Mj ; (2) no symbols in t appear in other triples in Mj ; and (3) the triple t
does not contradict the matching M 0. Note that this process is equivalent to
lling the missing symbols `' in the triple t(i j ) by the corresponding symbols
in the triple t.
The above process ends up with a matching M 0 in S . Note that the
matching M 0 can be constructed from the matching Mj in time O(n). If
jM 0j = k, i.e., if all missing symbols in Mj are lled, then we are done (note
150
that the matching M 0 may not necessarily be the matching M0). Otherwise,
jM 0j < k. Without loss of generality, suppose that the triples in Mj whose
missing symbols are lled are the triples t(1j ) , : : :, t(hj ) , h < k. Now consider
the triple t(hj+1
) , which corresponds to the triple t
h+1 in the matching M0 .
It is clear that the only reason that the triple th+1 was not included in the
matching M 0 is that the triple th+1 contradicts the matching M 0 . According
to the way we construct the matching M 0 , the symbols in th+1 that also
appear in t(hj+1) cannot be in M 0 . Thus, the symbols in t
h+1 that are in
( j )
M 0 must correspond to the symbol `' in th+1 . Now we guess a symbol
bh+1 in M 0 such that bh+1 is in th+1 and corresponds to a `' in t(hj+1 ) , and
replace the corresponding symbol `' in t(hj+1
) by b . This gives us a new
h+1
pseudo-matching
Mj +1 = ft(1j +1); t(2j+1); : : :; t(kj+1)g
where t(i j +1) = t(i j ) for all i 6= h + 1, and t(hj+1
+1) is the triple t(j ) with a
h+1
symbol `' replaced by the symbol bh+1 .
Therefore, both pseudo-matchings Mj and Mj +1 are the matching M0
with some symbols replaced by the symbol `'. Moreover, the pseudo-
matching Mj +1 has one less `' than the pseudo-matching Mj . It is clear
that the matching Mj +1 can be constructed from the matching Mj in time
O(n) if the guessed symbol bh+1 is given. Now our algorithm applies the
same process on the matching Mj +1 .
Since we started with the matching M2 with 2k `' symbols and the
above algorithm reduces the number of `' symbols by one from Mj to Mj +1 ,
the algorithm must end up with a matching Mg of k triples that contains
no `' symbols, where g  2k + 2. This completes the description of our
nondeterministic algorithm. Our nondeterministic algorithm runs in time
O(kn) if the guessed symbols are all given.
We point out that our nondeterministic algorithm reports a matching of
k triples in S only if it actually nds a matching of k triples. Therefore, if
a maximum matching in S contains less than k triples, then our nondeter-
ministic algorithm will be stuck at some point without having a matching
of k triples. An incorrect guess may also spoil the process. However, if
the maximum matching in S has at least k triples, and if all our guesses in
the process are correct, then the nondeterministic algorithm will produce a
matching of k triples.
Now we explain how the nondeterminism in the above algorithm can
151
be eliminated. Each guess in the algorithm corresponds to a sequence of
nondeterministic binary bits. We rst calculate how many nondeterministic
binary bits are needed in the algorithm.
In constructing the pseudo-matching M2 from the maximal matching
M1 , we need guess k symbols in M1 . Since M1 contains 3jM1 j < 3k symbols,
k symbols in M1 can be represented by a binary vector of length 3jM1j, in
which exactly k bits are 1. Therefore, guessing k symbols in M1 takes no
more than 3k nondeterministic binary bits.
When we construct the pseudo-matching Mj +1 from the pseudo-matching
Mj , we need to guess the symbol bh+1 from the matching M 0 . First we need
at most one nondeterministic binary bit to decide which `' symbol in t(hj+1)
should be lled (recall that t(hj+1
) contains at most two `' symbols). Once
the `' symbol in t(hj+1
) is decided, we only need to look at the triples in M 0
on the corresponding dimension. Since the matching M 0 contains less than
k triples, M 0 contains less than k symbols in each dimension. Therefore,
guessing a symbol in M 0 corresponding to the chosen `' in t(hj+1) is equiva-
lent to deciding a position out of jM 0j positions. Thus, guessing the symbol
bh+1 totally takes no more than 1 + log k nondeterministic binary bits.
Since the nondeterministic algorithm ends up with a matching Mg , with
g  2k + 2, we conclude that the total number of nondeterministic binary
bits used by the nondeterministic algorithm is bounded by (note that the
pseudo-matching starts from M2)
3k + 2k(1 + log k) = k(5 + 2 log k)
To convert the nondeterministic algorithm into a deterministic algorithm,
we run the nondeterministic algorithm using each of the 2k(5+2 log k) binary
vectors of length k(5 + 2 log k) as the k(5 + 2 log k) nondeterministic binary
bits. Since for a xed such binary vector, the algorithm runs in time O(kn),
we conclude that the running time of the resulting deterministic algorithm
is bounded by O(n23k log k ).
Theorem 30.1 There is an algorithm A such that given a set S of n triples
and an integer k, the algorithm A runs in time O(n23k log k ), either nds a
matching of k triples in S or reports no such a matching exists in S .
Now we come back to approximation algorithms for the 3-D Match-
ing problem. In the last lecture, we presented an algorithm Apprx3D-First
that runs in linear time and constructs a maximal matching for a given set
152
of triples. We proved that the number of triples in a maximal matching is
at least 1=3 the number of triples in a maximum matching (Theorem 29.2).
Thus, the algorithm Apprx3D-First is an approximation algorithm of ap-
proximation ratio 3 for the 3-D Matching problem. Now we present an-
other polynomial time approximation algorithm with a better approximation
ratio for the 3-D Matching problem.
Let S be a set of triples and let M be a maximal matching in S . Since
the matching M is maximal, no triple in S M can be added directly to
M to obtain a larger matching. However, it is possible that if we remove
one triple from M , then we are able to add two triples from S M to M
to obtain a larger matching. We say that the matching M is 1-optimal if
no such a triple in M exists. More formally, we say that a matching M is
1-optimal if M is maximal and it is impossible to nd a triple t1 in M and
two triples t2 and t3 in S M such that M ft1 g [ ft2; t3 g is a matching
in S .
We present an algorithm that constructs a 1-optimal matching for a
given set of triples.
Algorithm 30.1 Apprx3D-Second
Input: a set of S n triples
Output: a matching M in S
1. construct a maximal matching M
using Apprx3D-First;
2. change = true;
3. while change do
for each triple in t M do
M M ftg
= ;
S
let t be the set of triples not contradicting ; M
construct a maximum matching t in Mt ; S
if M t contains more than one triple
then M M [ M= t ; change = true;
else M M [ ftg
= ;

Lemma 30.2 After each execution of the for loop in step 3 of the algorithm
Apprx3D-Second, the matching M is a maximal matching.
proof. Before the algorithm enters step 3, the matching M is maximal.
Since the set St has no common symbol with the matching M after the
triple t is removed from M , for any matching M 0 in St , M [ M 0 is a matching
in S . Moreover, since all triples in S St contradict M , and all triples in
153
St Mt contradict Mt, we conclude that all triples in S (M [ Mt ) contradict
M [ Mt. That is, the matching M [ Mt is a maximal matching in S , which
is assigned to M if Mt has more than one triple. In case Mt has only one
triple, the triple t is put back to M , which by induction is also maximal.
Lemma 30.3 The matching constructed by the algorithm Apprx3D-Second
is 1-optimal.
proof. It is easy to see that there are a triple t in M and two triples t1 and
t2 in S M such that M ftg [ ft1; t2 g is a matching in S if and only if the
matching Mt in St contains more than one triple. Therefore, the algorithm
Apprx3D-Second actually goes through all triples in M and checks whether
each of them can be traded for more than one triple in S M . In other
words, the algorithm Apprx3D-Second ends up with a 1-optimal matching
M.
Lemma 30.4 The maximum matching Mt in the set St can be constructed
in time O(n).
proof. We rst show that a maximum matching in St contains at most
3 triples. Suppose that t1 , t2 , t3 , and t4 are four triples in a maximum
matching in St . Then at least one of them, say t1 , contains no symbol in
the triple t. Since t1 does not contradict M ftg, t1 does not contradict M
even before t is removed from M . Therefore, before the triple t is removed,
the matching M is not maximal. This contradicts Lemma 30.2.
Therefore, a maximum matching in St contains at most 3 triples. Now
according to Theorem 30.1, we can nd a maximum matching Mt in St in
time O(n).
Since each execution of the while loop in algorithm Apprx3D-Second
increases the number of triples in M by at least 1 and a maximum matching
in S contains at most n triples, we have the following theorem.
Theorem 30.5 The algorithm Apprx3D-Second runs in time O(n3).
We analyze the approximation ratio for the algorithm Apprx3D-Second.
Theorem 30.6 The algorithm Apprx3D-Second has approximation ratio 2.
proof. We denote by M the matching in S constructed by the algorithm
Apprx3D-Second and let Mmax be a maximum matching in S .
154
 Based on the matchings M and Mmax, we introduce a weighting function
w(
) on symbols in X [ Y [ Z as follows.
 if a symbol a is not in both M and Mmax, then the symbol a has weight
0: w(a) = 0;
 if a symbol a is in both M and Mmax, and a is in a triple of Mmax that
contains two symbols not in M , then a has weight 1: w(a) = 1;
 if a symbol a is in both M and Mmax, and a is in a triple of Mmax that
contains only one symbol not in M , then a has weight 1=2: w(a) = 1=2;
 if a symbol a is in both M and Mmax, and a is in a triple of Mmax that
contains no symbol not in M , then a has weight 1=3: w(a) = 1=3;
The weight w(t) of a triple t is the sum of the weights of its components.
According to the denition, each triple in the matching Mmax has weight
exactly 1.
Now let t = (a; b; c) be a triple in M . If w(t) > 2, then at least two
components of t have weight 1. Without loss of generality, suppose that
w(a) = w(b) = 1. By the denition, there are two triples t1 = (a; b0; c0)
and t2 = (a00; b; c00) in the matching Mmax such that the symbols b0, c0,
a00, c00 do not appear in M . However, this would imply that M ftg [
ft1; t2g is a matching and the matching M constructed by the algorithm
Apprx3D-Second would not be 1-optimal. This contradicts Lemma 30.3.
Thus, each triple in the matching M has weight at most 2. Since only
symbols in both matchings M and Mmax have nonzero weight, we must have
X X
w(t) = w(t)
t2Mmax t2M
P
Since each triple in Mmax has weight 1, we have t2Mmax w(P t) = jMmaxj.
Moreover, since each triple in M has weight at most 2, we have t2M w(t) 
2jM j. This gives us
jMmaxj  2jM j
or jMmaxj=jM j  2. This completes the proof.
Corollary 30.7 The 3-D Matching problem has an approximation algo-
rithm that runs in time O(n3 ) and has approximation ratio 2.

155
Remark 30.1 A natural extension of the algorithm Apprx3D-Second is to
consider 2-optimal, or in general k-optimal. That is, we construct a maximal
matching M in S such that no k triples in M can be traded for k + 1 triples
in S M . It is not very hard to see that a k-optimal matching in S can be
constructed in polynomial time when k is a xed constant. In fact, using
Theorem 30.1, we can develop an algorithm of running time O(nk+2 ) that
constructs a k-optimal matching for a set S of n triples. We can show that a
k-optimal matching gives an approximation ratio smaller than 2 when k > 1.
For example, a 2-optimal matching has approximation ratio 9=5 while a 3-
optimal matching has approximation ratio 5=3. It can also been shown that
the approximation algorithm for the 3-D Matching problem by constructing
k-optimal matchings for a xed constant k cannot have approximation ratio
less than or equal to 3=2.
Course Project Problem: Develop an approximation algorithm for the
3-D Matching problem that uses an approach dierent from the k-optimality
method and has approximation ratio better than 2.

156
 CPSC-669 Computational Optimization
Lecture #31, November 8, 1995
Lecturer: Professor Jianer Chen
Scribe: Shijin Lu
Revision: Jianer Chen
31 Maximum satisability
We have studied the Vertex Cover problem and the 3-D Matching
problem. These two problems have a common property that both of them
have polynomial time approximation algorithms whose approximation ratio
is bounded by a constant c > 1. It is unknown how close this constant c can
be to the value 1. In particular, do these problems have a polynomial time
approximation scheme? Very recent progress in computational optimization
has shown that a large class of optimization problems of practical impor-
tance falls into this category. For this reason, we introduce another class of
optimization problems.
Denition 31.1 An optimization problem is approximable with a constant
ratio in polynomial time if it has a polynomial time approximation algorithm
with approximation ratio c, where c is a xed constant. Let APX be the
class of all optimization problems approximable with a constant ratio in
polynomial time.
It is clear that the class PTAS is a subclass of the class APX.
It is well-known that the Satisfiability problem plays a fundamental
role in the study of NP-completeness. An optimization version of the Satis-
fiability problem, the Max-Sat problem, plays a similar role in the study
of the optimization class APX.
Let X = fx1 ; : : :; xn g be a set of boolean variables. A literal in X is
either a boolean variable xi or its negation xi , for some 1  i  n. A clause
on X is an Or of a set of literals in X . The Satisfiability problem is
formally dened as follows.
Satisfiability (Sat)
Input: a set of clauses C1; C2; : : :; Cm on fx1; : : :; xng
Question: does there exist a truth assignment on fx1 ; : : :; xng
that satises all clauses?
157
 By the famous Cook's Theorem, the Satisfiability problem is NP-
complete.
If we have further restrictions on the number of literals in each clause,
we obtain another two interesting complexity classes.
3-Satisfiability (3Sat)
Input: a set of clauses C1; C2; : : :; Cm on fx1 ; : : :; xng such
that each clause has exactly 3 literals
Question: does there exist a truth assignment on fx1 ; : : :; xng
that satises all clauses?
2-Satisfiability (2Sat)
Input: a set of clauses C1; C2; : : :; Cm on fx1 ; : : :; xng such
that each clause has exactly 2 literals
Question: does there exist a truth assignment on fx1 ; : : :; xng
that satises all clauses?
It is well-known that the 3-Satisfiability problem is still NP-complete,
while the 2-Satisfiability problem can be solved in polynomial time (in
fact, in linear time).
An optimization version of the Satisfiability problem can be dened
as follows.
Max-Sat
Input: a set of clauses C1; C2; : : :; Cm on fx1; : : :; xng
Output: a truth assignment on fx1; : : :; xng that satises the
maximum number of the clauses
The optimization versions for the 3-Satisfiability problem and for the
2-Satisfiability problem are
Max-3Sat
Input: a set of clauses C1; C2; : : :; Cm on fx1 ; : : :; xng such
that each clause has at most 3 literals
Output: a truth assignment on fx1; : : :; xng that satises the
maximum number of the clauses

158
 Max-2Sat
Input: a set of clauses C1; C2; : : :; Cm on fx1 ; : : :; xng such
that each clause has at most 2 literals
Output: a truth assignment on fx1; : : :; xng that satises the
maximum number of the clauses
It is easy to see that the Satisfiability problem can be reduced in poly-
nomial time to the Max-Sat problem: an instance fC1; : : :; Cmg is a Yes-
instance for the Satisfiability problem if and only if when it is regarded
as an instance of the Max-Sat problem, its optimal value is m. Therefore,
the Max-Sat problem is NP-hard. Similarly, the 3-Satisfiability prob-
lem can be reduced in polynomial time to the Max-3Sat problem so that
the Max-3Sat problem is NP-hard.
Since the 2-Satisfiability problem can be solved in linear time, one
may expect that the corresponding optimization problem Max-2Sat is also
easy. However, the following theorem gives a bit surprising result.
Lemma 31.1 The Max-2Sat problem is NP-hard.
proof. We show that the 3-Satisfiability problem can be reduced in
polynomial time to the Max-2Sat problem.
Let E = fC1; : : :; Cmg be an instance for the 3-Satisfiability problem,
where each Ci is a clause of three literals in fx1 ; : : :; xng. Consider the
clause Ci = (ai _ bi _ ci ), where ai , bi, and ci are literals in fx1; : : :; xng. We
construct ten clauses:
Ci = f(ai ); (bi); (ci); (yi);
(ai _ bi); (ai _ ci ); (bi _ ci); (11)
(ai _ yi ); (bi _ yi ); (ci _ yi )g
where yi is a new created boolean variable. It is easy to verify the following
facts.
 if none of ai, bi, ci is true, then any assignment to yi can make at most
six out of the ten clauses in (11) true;
 if one of ai, bi, ci is true and two of ai, bi, ci are false, then no as-
signment to yi can make more than seven of the ten clauses in (11)
true and there is an assignment to yi that makes seven out of the ten
clauses in (11) true;
159
  if two of ai, bi, ci are true and one of ai, bi, ci is false, then no as-
signment to yi can make more than seven of the ten clauses in (11)
true and there is an assignment to yi that makes seven out of the ten
clauses in (11) true;
 if all ai, bi, ci are true, then no assignment to yi can make more than
seven of the ten clauses in (11) true and there is an assignment to yi
that makes seven out of the ten clauses in (11) true;
Based on the above analysis, we conclude that if we set any of the three
literals ai , bi , and ci true, then no assignment to yi can make more than
seven of the ten clauses in (11) true and there is an assignment to yi that
makes seven out of the ten clauses in (11) true, and if we set all three literals
ai , bi, ci false, then any assignment to yi can make at most six out of the
ten clauses in (11) true.
Now let E be the set of 10m clauses in C1, : : :, Cm , where each Ci is
given as in (11). E is an instance for the Max-2Sat problem. It is easy
to see that the instance E can be constructed in polynomial time from the
instance E .
Suppose that E is a Yes-instance for the 3-Satisfiability problem.
Then there is an assignment Sx to fx1; : : :; xn g that makes at least one
literal in each Ci of the clauses C1, : : :, Cm true. According to the analy-
sis given above, this assignment Sx together with a proper assignment Sy
to fy1; : : :; ym g will make seven out of the ten clauses in Ci true, for all
i = 1; : : :; m. Therefore, the assignment Sx + Sy to the boolean variables
fx1; : : :; xn; y1; : : :; ymg makes 7m clauses in E true. This gives Opt(E) 
7m.
Now suppose that E is a No-instance for the 3-Satisfiability prob-
lem. Let S be an assignment to fx1; : : :; xn ; y1; : : :; ymg and we analyze how
many clauses in E the assignment S can satisfy. The assignment S can be
decomposed into an assignment Sx to fx1 ; : : :; xng and an assignment Sy to
fy1; : : :; ymg. Since E is a No-instance for the 3-Satisfiability problem,
for at least one clause Ci in E , the assignment Sx makes all literals false.
According to our previous analysis, any assignment to yi together with the
assignment Sx can make at most six out of the ten clauses in Ci true. More-
over, since no assignment to fx1; : : :; xn ; y1; : : :; ym g can make more than
seven clauses in each Cj true, we conclude that the assignment S can make
at most 7(m 1) + 6 = 7m 1 clauses in E true. Since S is arbitrary,
we conclude that in this case, no assignment to fx1 ; : : :; xn; y1; : : :; ym g can
make more than 7m 1 clauses in E true.
160
 Summarizing the discussion above, we conclude that E is a Yes-instance
for the 3-Satisfiability problem if and only if the optimal value Opt(E ) for
the instance E for the Max-2Sat problem is equal to 7m. Consequently, the
3-Satisfiability problem can be reduced in polynomial time to the Max-
2Sat problem. This completes the proof that the Max-2Sat problem is
NP-hard.
Now we describe an approximation algorithm for the Max-Sat problem.
Consider the following algorithm. For each clause, we give it a weight w(Ci).
We use jCij to denote the number of literals in the clause Ci .
Algorithm 31.1 ApprxMaxSat
Input: a set of clauses 1 fC ; : : :; Cmg on fx1; : : :; xng
Output: a truth assignment to fx1; : : :; xng
1. LEFT = fC ; : : :; Cmg;
1
2. for each clause Ci do
w(Ci) = 1=2jC j
i

3. for i = 1 to n do
find all clauses C1t ; : : :; Crt in LEFT that contain xi ;
P P
find all clauses C1f ; : : :; Csf in LEFT that contain xi ;
if i=1 w(Cit) > j=1 w(Cjf )
r s
then xi = 1
delete C1t ; : : :; Crt from LEFT;
for j = 1 to s do w(Cjf ) = 2w(Cjf )
else xi = 0 f
delete C1 ; : : :; Crf from LEFT;
for j = 1 to r do w(Cjt) = 2w(Cjt)
The algorithm ApprxMaxSat runs in polynomial time. Now we analyze
the approximation ratio for the algorithm.
Lemma 31.2 If each of the input clauses fC1; : : :; Cmg contains at least
k literals, then the algorithm ApprxMaxSat constructs an assignment that
satises at least m(1 1=2k ) of the clauses.
proof. In the algorithm ApprxMaxSat, once a clause is satised, the
clause is deleted from the set LEFT. Therefore, the number of clauses that
are not satised by the constructed assignment is equal to the number of

161
clauses left in the set LEFT at the end of the algorithm. We calculate the
number of clauses in LEFT using the weighting function w(
).
Initially, each clause Ci has weight 1=2jCij . By our assumption, the clause
Ci contains at least k literals. So we have
X X X
w(Ci) = 1=2jCij  1=2k = m=2k
Ci 2LEFT Ci 2LEFT Ci 2LEFT
In step 3, we update the set LEFT and the weight forP the clauses in LEFT. It
can be easily seen that we never increase the value Ci 2LEFT w(Ci) | each
time we update the set LEFT, we delete a heavier set of clauses in LEFT and
double the weight for a lighter set of clauses in LEFT. Therefore, at end of
the algorithm, we should have
X
w(Ci)  m=2k (12)
Ci 2LEFT
At the end of the algorithm, all boolean variables fx1 ; : : :; xng have been
assigned a value. A clause Ci in the set LEFT has been considered by the
algorithm exactly jCij times and each time the corresponding literal in Ci
was assigned 0. Therefore, for each literal in Ci, the weight of the clause
Ci is doubled once. Since initially the clause Ci has weight 1=2jCij and its
weight is doubled exactly jCij times in the algorithm, we conclude that at
the end of the algorithm, the clause Ci in LEFT has weight 1. Combining this
with the inequality (12), we conclude that at the end of the algorithm, the
number of clauses in the set LEFT is no more than m=2k . In other words,
the number of clauses satised by the constructed assignment is at least
m(1 1=2k ). The lemma is proved.
Theorem 31.3 For an input of m clauses each containing at least k > 0
literals, the algorithm ApprxMaxSat constructs an assignment with approx-
imation ratio 1 + 1=(2k 1). In particular, the algorithm ApprxMaxSat is
an approximation algorithm with approximation ratio 2 for the Max-Sat
problem.
proof. According to Lemma 31.2, on an input of m clauses each contain-
ing at least k literals, the algorithm ApprxMaxSat constructs an assignment
that satises at least m(1 1=2k ) clauses. Since no assignment can satisfy
more than m clauses, the approximation ratio must be bounded by
m = 1 + 1
m(1 1=2 ) k k
2 1
162
 Since for each input instance for the Max-Sat problem, each clause con-
tains at least 1 literal, the second statement in the theorem follows directly.

Remark 31.2 The approximation ratio 2 for the Max-Sat problem is due
to a classical work of David Johnson about 20 years ago. The bound 2 stood
for more than 20 years until recently, Yannakakis developed a polynomial
time approximation algorithm with approximation ratio 4=3 for the Max-
Sat problem.

163
 CPSC-669 Computational Optimization
Lecture #32, November 10, 1995
Lecturer: Professor Jianer Chen
Scribe: Hao Zheng
Revision: Jianer Chen
32 Probabilistically Checkable Proofs
We have seen a number of optimization problems that are in APX, that
is, that have polynomial time approximation algorithms with approxima-
tion ratio bounded by a constant: the
-Traveling Salesman problem
can be approximated in polynomial time with approximation ratio 1:5, the
Vertex Cover problem can be approximated in polynomial time with ap-
proximation ratio 2, the 3-D Matching problem can be approximated in
polynomial time with approximation ratio 2, and the Max-Sat problem
can be approximated in polynomial time with approximation ratio 2. The
ratios for the rst two problems are still the best results known today, and
the ratios for the last two problems have be somehow improved recently to
a constant c > 1 (c = 1:5 +  for the 3-D Matching problem and c = 1:325
for the Max-Sat problem). The question is whether further improvement
on the approximation ratio is possible. In particular, how close can this
approximation ratio be to the value 1? Can they have a polynomial time
approximation scheme?
The questions turn out to be very deep in the study of computational
optimization. We will see later that from a viewpoint of complexity theory,
these questions are equivalent to the famous P = NP problem. Moreover, our
algorithmic practice also suggests the possibility for either directions. Take
the
-Traveling Salesman problem as an example. It has been more
than 15 years that the bound 1:5 on the approximation ratio has stood for
the problem. On the other hand, very recent research (still in manuscript
version) has shown that the Graph Traveling Salesman problem, in
which the distance metric between two vertices is the shortest path met-
ric of an unweighted graph, has a polynomial time approximation scheme.
Note that the Graph Traveling Salesman problem is a restricted version
of the
-Traveling Salesman problem. From this progress, researchers
have even conjectured that the Euclidean Traveling Salesman prob-

164
lem, which seems the most naturally restricted version of the
-Traveling
Salesman problem, has a polynomial time approximation scheme.
Researchers were not able to answer these questions for more than 20
years until a very recent breakthrough in complexity theory that gives a new
characterization of the complexity class NP. In the rest of this lecture, we
will describe this new characterization.
We need to review a few fundamental denitions in complexity theory.
Denition 32.1 A language L is a subset of , where  is a xed alphabet.
With a proper coding scheme, we can assume that  = f0; 1g. For an
instance x 2 , if x 2 L then we say that x is a Yes-instance of L while if
x 62 L then we say that x is a No-instance of L.
A language L is also called a \decision problem" in which for each in-
stance x, we need to decide a \Yes/No" conclusion for the question \x 2 L?"
Denition 32.2 A language L is accepted by an algorithm A if on any
input instance x 2 , the algorithm A outputs \Yes" if x 2 L (or we say
that A accepts x), and \No" if x 62 L (or we say that A rejects x).
Denition 32.3 An algorithm A is nondeterministic if it works as follows:
on an input instance x 2  , the algorithm A is also provided with another
\guessed" string yx 2  (by some magic way). Thus, the algorithm A can
work on x with the \hints" given in the guessed string y . The nondetermin-
istic algorithm A accepts a language L if for each x 2 L, there is a guessed
string yx such that A accepts x when yx is provided, and for each x 62 L,
the algorithm A rejects x for any guessed string y .
A nondeterministic algorithm A runs in polynomial time if the running
time of A is bounded by a polynomial of the input length jxj. Note that the
time complexity of a nondeterministic algorithm is not measured in terms
of the guessed string y . Since a polynomial time nondeterministic algorithm
A can read at most polynomial many bits in y , we can assume, without
loss of generality, that the length of the guessed string y is bounded by a
polynomial of the input length jxj.
We say that an algorithm A reduces a language L1 to another language
L2 if on any input instance x1 for L1 , the algorithm A produces an input
instance x2 for L2 such that x1 2 L1 if and only if x2 2 L2 .

165
Denition 32.4 A language L is in NP if it is accepted by a polynomial
time nondeterministic algorithm. A language L is NP-complete if L is in
NP and for each language L0 in NP, there is a polynomial time algorithm
that reduces L0 to L.
Take the Satisfiability problem as an example. Given a set x of clauses
C1, : : :, Cm on fx1; : : :; xng, a polynomial time nondeterministic algorithm
A can work as follows: A interprets the rst n binary bits in the guessed
string y as a truth assignment to the boolean variables fx1 ; : : :; xng and
replaces each literal in the clauses by the corresponding boolean value. A
accepts x if all clauses are evaluated true on this assignment, otherwise A
rejects x. It is easy to see that if x is a Yes-instance for the Satisfiability
problem, then for the guessed string yx whose rst n bits give the assignment
that satises x, the algorithm A will accept. On the other hand, if x is a
No-instance for the Satisfiability problem, then no matter which guessed
string y is provided, the algorithm A will reject anyway since no assignment
on fx1; : : :; xn g can satisfy all the clauses in x. Therefore, the Satisfiabil-
ity problem is in NP. By the famous Cook's theorem, the Satisfiability
problem is actually NP-complete.
A nondeterministic algorithm A accepting a language L can also be
interpreted as a proof system. The given input instance x can be regarded
as a statement of a theorem \the string x is in the language L," while the
guessed string y can be regarded as a proof for the theorem. The algorithm
A is a very trusty \verier", who may not be able to derive a proof for
the theorem x, but can verify whether y is a valid proof for the theorem x.
Therefore, if the theorem x is true and the guessed string y is a valid proof
for the theorem x, then the algorithm A will say \Yes", and if the theorem
x is not true then the algorithm A will disprove any pseudo-proof y and say
\No". In this sense, each problem in NP is a set of theorems that have valid
proofs that are \easily checkable", i.e., that can be checked in polynomial
time.
An interesting question is how many bits of the proof y a polynomial time
nondeterministic algorithm needs to read in order to verify the theorem x.
In real life, it seems that most of the theorems simply need a single \hint"
and the other parts of the proof can be easily derived from the hint. Is
this also true for the problems in NP? For this, we introduce the following
denition.
Denition 32.5 A language L is in the class PCP(0; b(n)) if L is accepted
by a polynomial time nondeterministic algorithm A such that on each input
166
instance x, the algorithm A reads at most O(b(jxj) bits from the guessed
string y .
We have the following easy observations.
Lemma 32.1 Every language in NP is in the class PCP(0; nc) for some
constant c.
proof. Suppose that a language L is in NP. Then L is accepted by a
polynomial time nondeterministic algorithm A. Let the running time of the
algorithm A be O(nc ). Then the algorithm A on input instance x reads at
most O(jxjc) bits from a guessed string y . That is, the language L is in the
class PCP(0; nc).
It is interesting to ask whether it is possible to have a polynomial time
algorithm that accepts a language in NP, in particular an NP-complete lan-
guage, by reading less than
(n) bits from the guessed string y . The con-
jecture is No. However, it seems that our current knowledge is still far from
a formal proof of this conjecture. A (much) weaker result can be formally
proved: if a language is in PCP(0; log log n), then it is in P.
Another extension of our deterministic algorithms is probabilistic algo-
rithms, dened as follows.
Denition 32.6 An algorithm A is a probabilistic algorithm if on any in-
put instance x, the algorithm A rst generates a random string r, then
deterministically works on the input x.
If the outcome of a probabilistic algorithm does not depend on the gener-
ated random string, then the probabilistic algorithm is just a normal deter-
ministic algorithm. If the computation of the probabilistic algorithm does
depend on the generated random string, then each outcome of the computa-
tion will happen with a certain probability. Some very interesting practical
problems can be solved by probabilistic algorithms in such a way that correct
solutions are produced by the algorithm with very high probability.
If we allow both probabilism and nondeterminism, we obtain the follow-
ing class.
Denition 32.7 A language L is in the class PCP(r(n); b(n)) if it is ac-
cepted by a polynomial time algorithm A with two constants c and d such
that
167
 1. On an input instance x, the random string generated by the algorithm
A is of length c
r(jxj)), a guessed string y of length O(jxjd) is provided,
and the algorithm A reads at most O(b(jxj)) bits from a guessed string;
2. For each input instance x 2 L, there is a guessed string yx of length
O(jxjd) such that the algorithm A accepts x with probability 1 (i.e.,
A accepts x based on yx for every generated random string of length
c
r(jxj));
3. For each input instance x 62 L, on any guessed string y of length
O(jxjd), the algorithm A rejects x with probability at least 1=2 (i.e., A
rejects x based on y for at least half of the generated random strings
of length c
r(jxj)).
The algorithm A is called a PCP(r(n); b(n)) system accepting the language
L.
The name \PCP" here refers to the \probabilistically checkable proof"
as the model involves a checkable proof system (i.e., guessed strings) and
probabilistic computation (i.e., the random string generation).
It was a very active research topic that for what functions r(n) and b(n),
the class PCP(r(n); b(n)) precisely describes the class NP. The question was
eventually settled down recently, as a result stated as follows.
Theorem 32.2 A language L is in the class NP if and only if L is in the
class PCP(log n; 1).
The current proof for Theorem 32.2 is rather involved. It borrows sig-
nicantly from results on polynomial checking, proof verication, program
result checking, and coding theory. Giving the details of these results goes
far beyond the scope of this course and we refer the interested students to
the papers that originate them (talk to the instructor).

168
 CPSC-669 Computational Optimization
Lecture #33, November 13, 1995
Lecturer: Professor Jianer Chen
Scribe: Hao Zheng
Revision: Jianer Chen
33 Max-3Sat has no PTAS
In the previous lecture, we dened the PCP systems. The following theorem
was stated.
Theorem 33.1 A language L is in the class NP if and only if L is accepted
by a PCP(log n; 1) system.
An outstanding application of Theorem 33.1 is a proof that many opti-
mization problems, such as Max-3Sat, have no polynomial time approxi-
mation scheme unless P = NP. Before we present the proof, we rst make
a closer look at the PCP systems. This investigation should let us have a
better understanding on the PCP systems.
By the denition, a PCP(log n; 1) system is a polynomial time algorithm
A that on input of length n, generates a random string of length O(log n) and
reads at most b bits from the guessed string y , where b is a xed constant.
It should be noted that which b bits of the guessed string y are read by the
algorithm A may depend on the values of the bits read from y . For example,
suppose that the algorithm A reads the rst bit from the guessed string y .
Now the algorithm A may calculate the address of the second bit to be read
from y based on the value of the rst bit. In general, the address of the ith
bit to be read by A from y may depend on the values of the rst i 1 bits
read by A.
Denition 33.1 A PCP(r(n); b(n)) system A is nonadaptive if on an input
instance x and a xed randomly generated string of length O(r(jxj)), the
addresses of the O(b(n)) bits to be read by A from the guessed string y are
independent of the content of the guessed string y .
Therefore, the process of a nonadaptive PCP(r(n); b(n)) system A can
be regarded as follows: on input instance x, the polynomial time algorithm

169
A rst generates a random string of length O(r(jxj)), then generates the
O(b(n)) addresses for the bits to be read from the guessed string y , and
then reads the bits from y . There will be no other computation performed
during the reading of the bits from the guessed string y .
Lemma 33.2 A language L is accepted by a PCP(log n; 1) system if and
only if it is accepted by a nonadaptive PCP(log n; 1) system.
proof. By the denition, if the language L is accepted by a nonadaptive
PCP(log n; 1) system, then L is accepted by a general PCP(log n; 1) system.
Now suppose that the language L is accepted by a PCP(log n; 1) system
A1. By the denition, on each input instance x, the polynomial time algo-
rithm A1 rst generates a random string R of length O(log n), then works
on x based on R and a guessed string y . The algorithm A1 reads at most b
bits from the guessed string y , where b is a xed constant. If x 2 L, then
there is a guessed string yx such that on all randomly generated strings R
of length O(log n), A1 accepts x based on yx ; and if x 62 L, then for any
guessed string y , for at least half of the randomly generated strings of length
O(log n), the algorithm A1 rejects x based on y .
We construct a nonadaptive PCP(log n; 1) system A2 that accepts the
language L.
Let x be an input instance of L. Fix a randomly generated string R of
length O(log n). The algorithm A2 works as follows. A2 rst enumerates all
the 2b boolean vectors of length b. Note that each boolean vector (v1 ; : : :; vb )
of length b gives a possible set of values for the b bits to be read from the
guessed string y . For the xed input instance x and the xed random string
R, the boolean vector (v1; : : :; vb ) also uniquely determines the addresses
of the bits to be read by the algorithm A1 from the guessed string y : the
address of the rst bit depends only on x and R, the address of the second
bit depends on x, R, and the value of the rst bit, which is supposed to
be v1, and the address of the third bit depends on x, R, and the values of
the rst two bits, which are supposed to be v1 and v2, respectively, and so
on. Therefore, based on the input instance x, the random string R, and the
given boolean vector (v1 ; : : :; vb) of length b, the algorithm A2 can uniquely
determine the addresses of the b bits to be read by the algorithm A1 . The
algorithm A2 simulates the algorithm A1 on this computation, records the b
addresses on the guessed string, and records the decision of A1 on x and R
based on this boolean vector. Note that so far, no bits have been actually
read from the guessed string y , the values of the bits on the guessed string

170
y are assumed in the boolean vector (v1; : : :; vb).
The algorithm A2 performs the above operation on each of the 2b boolean
vectors of length b. At the end, the algorithm A2 has recorded d  b2b
addresses on the guessed string y . Now the algorithm A2 reads all these
d bits at once from the guessed string y . With these d values available,
the algorithm A2 can easily decide precisely which boolean vector gives the
correct sequence of values of the bits read from y by the algorithm A1 on
x and R. Note that there is exactly one such vector. With this correct
boolean vector, now the algorithm A2 can nd out whether the algorithm
A1 accepts x on the random string R and the guessed string y . The algorithm
A2 accepts x on R if and only if the algorithm A1 accepts x on R.
Now we can describe the algorithm A2 in a complete version. Given an
input instance x for L, the algorithm A2 simulates the algorithm A1 by rst
generating a random string R of length O(log n). Then, as described above,
the algorithm A2 simulates the algorithm A1 on x and R, and A2 accepts if
and only if A1 accepts.
By the construction, the algorithm A2 is clearly nonadaptive. The num-
ber of bits read by A2 from the guessed string y is bounded by d  b2b,
which is still a constant. The running time of A2 is bounded by b2b times
the running time of the algorithm A1 . Thus, A2 is also a polynomial time
algorithm. Finally, on any input instance x, any randomly generated string
R, and any guessed string y , the algorithm A2 makes the same decision as
the algorithm A1 . Thus, the algorithm A2 is a nonadaptive PCP(log n; 1)
system that accepts the language L.
Now we are ready for our main theorem in this lecture.
Theorem 33.3 The Max-3Sat problem has no polynomial time approxi-
mation scheme unless P = NP.
proof. Let L be any language that is NP-complete. We show that if the
Max-3Sat problem has a polynomial time approximation scheme, then the
language L can be solved in polynomial time by a deterministic algorithm,
which implies that P = NP.
Since the language L is in NP, by Theorem 33.1, L is accepted by a
PCP(log n; 1) system A. Without loss of generality, we assume that on
an input instance x, the polynomial time algorithm A generates a random
string R of length c log n, and reads at most b bits from a guessed string
y , where c and b are xed constants. If the input instance x is in L, then

171
there is a guessed string yx of length nd such that for all randomly generated
strings R of length c log n, the algorithm A accepts x; if the input instance
x is not in L, then for any guessed string y , the algorithm A rejects x on at
least half of the randomly generated strings R of length c log n. According to
Lemma 33.2, we can assume that the PCP(log n; 1) system A is nonadaptive.
Let x be a given input instance of L. Fix a random string R of length
c log n. We simulate the algorithm A on the input x and the random string
R. Note that the outcome of the algorithm A on input x and the random
string R depends on the values of the b bits to be read from the guessed
string y . Since the algorithm A is nonadaptive, the addresses i1, : : :, ib of
the b bits to be read from the guessed string y can be computed without
knowing the actual content of the guessed string y . Formally, the outcome
of the algorithm A on input x and the random string R is a boolean function
of b boolean variables:
Fx;R(yi1 ; : : :; yib )
where yij stands for the ij th bit of the guessed string y . The boolean function
Fx;R can be constructed by simulating the algorithm A on input x and
the random string R and on each of the possible assignments of yi1 , : : :,
yib . Note that there are only 2b dierent possibilities. Now convert the
function Fx;R (yi1 ; : : :; yib ) into the conjunctive normal form. Note that in
the conjunctive normal form, each clause has at most b literals and there
are at most 2b clauses. Now for each clause C = (z1 _ : : : _ za ) containing
more than 3 literals, we use the standard transformation, by introducing
a 3 new variables w1, : : :, wa 3, to convert it into a set of a 3 clauses of
3 literals:
(z1 _ z2 _ w1) ^ (w1 _ z3 _ w2) ^ (w2 _ z4 _ w3) ^
^


^ (wa 4 _ za 2 _ wa 3) ^ (wa 3 _ za 1 _ za)
After this transformation, the boolean function Fx;R has been converted into
a set Sx;R of clauses, in which each clause contains at most 3 literals. One
important fact is that the number of clauses contained in Sx;R is bounded
by b2b , a constant independent of the input instance x. Note that there is an
assignment to the boolean variables yi1 , : : :, yib that makes the function Fx;R
true if and only if there is an assignment to the variables in Sx;R that satises
all clauses in Sx;R . It is also clear that the set Sx;R can be constructed from
the input x and the random string R in polynomial time.
Now for each random string R of length c log n, we construct the set Sx;R
of clauses that contain at most 3 literals. The union of all these sets gives
172
us an input instance for the Max-3Sat problem:
[
S (x) = Sx;R
jRj=c log n
Since there are totally 2c log n = nc binary strings of length c log n, we can
rename the subsets Sx;R in S (x) as S1 , : : :, Sm , where m = nc :
S (x) = S1 [ S2 [


[ Sm
Since each set Si can be constructed in polynomial time and m = nc , the
set S (x) can be constructed from the input instance x in polynomial time.
Moreover, there is a constant h  b2b such that each subset Si contains at
most h clauses. Suppose that the set S (x) has N clauses. Then N  hm.
If the input x is in the language L, then according to the denition of
the PCP(log n; 1) system A, there is a guessed string yx of length nd such
that on every randomly generated string R, the algorithm A accepts x.
Consequently, if we let this yx be the assignment on the variables in the set
Sx;R, then all clauses in Sx;R are satised. That is, if we regard S (x) as an
input instance of the Max-3Sat problem, then the optimal value Opt(S (x))
of S (x) is N .
If the input instance x is not in the language L, then given any guessed
string y , for at least half of the random strings R of length c log n, the
algorithm A rejects x. That is, there are at least m=2 of the subsets S1 ,
: : :, Sm in S (x), for which the assignment y cannot satisfy all clauses in the
subset. Therefore, on any assignment to the variables in the set S (x), at
least m=2  N=(2h) clauses in the set S (x) are not satised.
In conclusion, the set S (x) is an input instance of the Max-3Sat problem
with the following properties:
Either x 2 L and there is an assignment to the boolean variables
in S (x) that satises all clauses in S (x), or x 62 L and no as-
signment to the boolean variables in S (x) can satisfy more than
N (1 1=(2h)) clauses in S (x), where h > 0 is a xed constant.
Moreover, the set S (x) of clauses can be constructed from x in
polynomial time.
Now it is straightforward to prove the theorem. Suppose that the Max-
3Sat problem has a polynomial time approximation scheme. Then let A0
be a polynomial time approximation algorithm with approximation ratio
1 + 1=(4h) for the Max-3Sat problem. We describe a polynomial time
173
deterministic algorithm A0 that accepts the language L. Given an input
instance x, the algorithm A0 rst constructs the instance S (x) in polynomial
time for the Max-3Sat problem. Suppose that S (x) has N clauses. The
algorithm A0 then applies the polynomial time approximation algorithm A0
on S (x) to produce a solution s for S (x), where s is an assignment to the
boolean variables in S (x). The algorithm A0 accepts x if and only if the
assignment s satises more than N (1 1=(2h)) clauses in S (x).
It is clear that the algorithm A0 is a polynomial time deterministic algo-
rithm. We prove that the algorithm A0 accepts precisely the language L. In
case x 2 L, by the above analysis, Opt(S (x)) = N . Since the approximation
algorithm A0 has approximation ratio 1+1=(4h), the assignment s produced
by the algorithm A0 must satisfy at least
N
1 + 1=(4h) > N (1 1=(2h))
clauses in S (x). In this case, the algorithm A0 accepts x. On the other
hand, if x 62 L, then by the above analysis, no assignment to the variables in
S (x) can satisfy more than N (1 1=(2h)) clauses in S (x). In particular, the
assignment s produced by the algorithm A0 satises no more than N (1
1=(2h)) clauses in S (x). Thus, in this case, the algorithm A0 rejects x.
This proves that the polynomial time algorithm A0 accepts precisely the
language L. We conclude that the NP-complete problem L is accepted by a
polynomial time deterministic algorithm. Consequently, P = NP.
This completes the proof.

174
 CPSC-669 Computational Optimization
Lecture #34, November 15, 1995
Lecturer: Professor Jianer Chen
Scribe: Hao Zheng
Revision: Jianer Chen
34 Independent Set has no PTAS
The famous Cook theorem that the Satisfiability problem is NP-complete
serves as a fundamental theorem for the study of NP-completeness of deci-
sion problems and gives the rst NP-complete problem. Because of this rst
NP-complete problem, the proofs for the NP-completeness of other decision
problems become much simpler by means of a proper \reduction" from the
Satisfiability problem.
Theorem 33.3 plays the same role in the study of approximability of opti-
mization problems as does Cook's theorem in the study of NP-completeness.
By Theorem 33.3, the hardness of approximability for the Max-3Sat prob-
lem is established. The hardness of approximability for other optimization
problems now can be established from the Max-3Sat problem by a proper
reduction that preserves the approximability. We will demonstrate a few
such reductions in this lecture. A formal denition of such a reduction
among optimization problems will be given in the next lecture.
The rst reduction is straightforward.
Lemma 34.1 If the Max-Sat problem has a polynomial time approxima-
tion scheme, then so does the Max-3Sat problem.
proof. Each instance x of the Max-3Sat problem is also an instance
for the Max-Sat problem. Therefore, any approximation algorithm for the
Max-Sat problem is also an approximation algorithm for the Max-3Sat
problem with the same approximation ratio. Therefore, if the Max-Sat
problem has a polynomial time approximation scheme, then so does the
Max-3Sat problem.
Theorem 34.2 The Max-Sat problem has no polynomial time approxima-
tion scheme unless P = NP.
proof. This follows directly from Theorem 33.3 and Lemma 34.1.

175
 Now let us consider a less simple reduction. Let G be a graph. Recall
that an independent set in G is a subset S of vertices in G in which no two
vertices are adjacent. The Independent Set problem is dened as follows.
Independent Set
Input: a graph G
Output: an independent set S of G with the cardinality of S
maximized
We present a reduction from the Max-3Sat problem to the Indepen-
dent Set problem.
The reduction is the one that is used in the NP-completeness theory to
show that the decision version of the Independent Set is NP-complete.
However, we need a more careful quantitative analysis.
Given an instance E = fC1; C2; : : :; Cmg of the Max-3Sat problem,
where each Ci is a clause of at most 3 literals in fx1; : : :; xn g. We construct
a graph GE as follows.
Every literal occurrence l in a clause Ci in E induces a vertex in the
graph GE , which will be named by l(i). Note that if the same literal appears
in two dierent clauses in E , then there will be two corresponding vertices
in the graph GE . For any pair of vertices l1(i) and l2(j ), there is an edge
connecting them if and only if either
1. i = j , i.e., the literals l1(i) and l2(j ) belong to the same clause in E ; or
2. i 6= j and l1(i) = l2(j ), i.e., the literals l1(i) and l2(j ) belong to dierent
clauses in E and they negate each other.
This completes the description of the graph GE .
Lemma 34.3 If  is an assignment to the variables fx1; : : :; xng that sat-
ises k clauses in E , then an independent set S of at least k vertices in the
graph GE can be constructed in polynomial time based on the assignment .
proof. Without loss of generality, suppose that the assignment  to the
variables fx1; : : :; xng satises the k clauses C1, : : :, Ck in E . Then under
this assignment , each clause Ci , i = 1; : : :; k, has (at least) one literal l(i)
that is set to true by . We claim that the subset S = fl(1); : : :; l(k)g of
vertices in the graph GE forms an independent set. In fact, for any pair of
vertices l(i) and l(j ), 1  i; j  k, i 6= j , since the literals l(i) and l(j ) belong
176
to dierent clauses Ci and Cj in E , the vertices l(i) and l(j ) are adjacent in
the graph GE only if the literal l(i) is the negation of the literal l(j ) . Thus,
any assignment will set one of the literals l(i) and l(j ) true and the other
false. By our assumption, the assignment  sets both the literals l(i) and
l(j) true. Thus, the literal l(i) cannot be the negation of the literal l(j) . In
consequence, the vertices l(i) and l(j ) in the graph GE are not adjacent. This
proves that the set S = fl(1); : : :; l(k)g is an independent set in the graph
GE . It is easy to see that the independent set S = fl(1); : : :; l(k)g in the
graph GE can be constructed in linear time when the assignment  to the
variables fx1; : : :; xn g is given.
Lemma 34.4 If the graph GE has an independent set S of k vertices, then
an assignment S to the variables fx1 ; : : :; xng can be constructed in poly-
nomial time such that S satises at least k clauses in E .
proof. Let S = fl1; : : :; lk g be an independent set in the graph GE . Since
no two vertices in S are adjacent, by the construction of the graph GE ,
1. no two literals li and lj in the set S belong to the same clause in E ;
and
2. no two literals li and lj in the set S negate each other. Thus, for each
variable xh , at most one of xh and xh is in S .
Thus, the set S = fl1; : : :; lk g induces an assignment S to the variables
fx1; : : :; xng such that S sets all literals l1, : : :, lk in S true. That is, if
xh is in S then S sets xh = 1 and if xh is in S then S sets xh = 0. For
variables xh such that neither of xh and xh appears in S , the assignment S
sets xh arbitrarily.
Now since the k literals l1, : : :, lk , which are set true by the assignment
S , belong to k dierent clauses in E , we conclude that the assignment S
satises at least k clauses in E . The assignment S to fx1 ; : : :; xn g can be
easily constructed in linear time from the independent set S in the graph
GE .
Corollary 34.5 The number of vertices in a maximum independent set in
the graph GE is equal to the maximum number of clauses in E that can be
satised by an assignment to fx1; : : :; xn g.
Now we are ready to prove:

177
Lemma 34.6 If the Independent Set problem has a polynomial time ap-
proximation scheme, then so does the Max-3Sat problem.
proof. Suppose that the Independent Set problem has a polynomial
time approximation scheme, we show how a polynomial time approximation
scheme for the Max-3Sat problem can be constructed.
For a given constant  > 0. Let ApxIS be a polynomial time approxi-
mation algorithm for the Independent Set problem with approximation
ratio 1 + . Consider the following algorithm.
Algorithm 34.1 Apx3Sat
Input: a set of clauses 1 fC ; : : :; C g
m , where each Ci is
a clause of at most 3 literals in 1 fx ; : : :; xng
Output: a truth assignment to 1 n fx ; : : :; x g
1. construct the graph E ;G
2. call the algorithm ApxIS on the graph E to find G
an independent set S
in E; G
3. construct an assignment S to 1 fx ; : : :; x g
n from S

such that S satisfies at least jS j
clauses in . E
It is clear that step 1 and step 2 of the algorithm Apx3Sat take polyno-
mial time. Lemma 34.4 proves that step 3 of the algorithm Apx3Sat also
takes polynomial time. Therefore, the algorithm Apx3Sat is a polynomial
time approximation algorithm for the Max-3Sat problem.
Now we analyze the approximation ratio for the algorithm Apx3Sat.
Let OptSAT (E ) be the optimal value of the set E = fC1; : : :; Cmg of
clauses, where E is treated as an instance of the Max-3Sat problem, and
let OptIS (GE ) be the optimal value of the graph GE , where GE is treated
as an instance of the Independent Set problem. By Corollary 34.5,
OptSAT (E ) = OptIS (GE )
Let Apx(S ) be the number of clauses in E that are satised by the assign-
ment S . According to step 3 of the algorithm Apx3Sat, we have
OptSAT (E )  OptSAT (E ) = OptIS (GE ) (13)
Apx(S ) jS j jS j
Since S is the independent set produced by the approximation algorithm
ApxIS for the Independent Set problem, by our assumption,
OptIS (GE )  1 + 
jS j
178
We conclude that the approximation ratio of the algorithm Apx3Sat for the
Max-3Sat problem is bounded by
OptSAT (E )  1 + 
Apx(S )
This proves that the polynomial time algorithm Apx3Sat is an approxima-
tion algorithm of approximation ratio 1 +  for the Max-3Sat problem.
Since  > 0 is arbitrary, we have proved that the Max-3Sat problem has a
polynomial time approximation scheme.
Theorem 34.7 The Independent Set problem has no polynomial time
approximation scheme unless P = NP.
proof. This follows directly from Lemma 34.6 and Theorem 33.3.
Another important optimization problem, Clique, is very closely related
to the Independent Set problem. Let G be a graph. A subset C of vertices
in G is a clique in G if all vertices in C are mutually adjacent.
Clique
Input: a graph G
Output: a clique C in G with the cardinality of C maximized
Let G = (V; E ) be a graph. The graph Gc = (V; E 0) with the same vertex
set V is called the complement graph of G if for any pair of vertices u and
v in V , u and v are adjacent in Gc if and only if u and v are not adjacent
in G. Note that the complement graph of the complement graph Gc is the
original graph G.
Lemma 34.8 Let G = (V; E ) be a graph and let Gc = (V; E 0) be the com-
plement graph of G. Let S be a subset of vertices in V . Then, S is an
independent set in the graph G if and only if S is a clique in the graph Gc .
proof. This follows directly from the denitions.
The approximabilities of the Clique problem and the Independent
Set problem are related by the following theorem.

179
Theorem 34.9 Let t(n) be a function that is at least as large as n. The
Clique problem has an approximation algorithm of running time O(t(n))
and approximation ratio r(n) if and only if the Independent Set problem
has an approximation algorithm of running time O(t(n)) and approximation
ratio r(n)).
proof. Let A-Clique be an approximation algorithm for the Clique
problem such that A-Clique has running time O(t(n)) and approximation
ratio r(n). Consider the following algorithm for the Independent Set
problem:
Algorithm 34.2 A-IS
Input: a graph G
Output: an independent set S in G
1. construct the complement graph c ; G
2. call the algorithm A-Clique on the graph c to find G
a clique S
in c ; G
3. return Sas an independent set in the graph . G
By Lemma 34.8, the set S constructed by the algorithm A-IS is an inde-
pendent set in the graph G. It is clear that the running time of the algo-
rithm A-IS is bounded by the running time of the algorithm A-Clique plus
O(n). By our assumption, the algorithm A-Clique runs in time O(t(n)) and
t(n) =
(n), thus we conclude that the running time of the algorithm A-IS
is also bounded by O(t(n)).
Again by Lemma 34.8, the number of vertices in a maximum indepen-
dent set in the graph G is equal to the number of vertices in a maximum
clique in the graph Gc . Therefore, if we let Optcl (Gc ) be the optimal value
for the graph Gc treated as an instance for the Clique problem and let
Optis (G) be the optimal value for the graph G treated as an instance for
the Independent Set problem, then we have
Optis (G) = Optcl (Gc)
jS j jS j
By our assumption, the approximation ratio for the approximation algo-
rithm A-Clique is bounded by r(n), thus Optcl (Gc )=jS j  r(n). This gives
immediately that the approximation ratio of the approximation algorithm
A-IS is also bounded by r(n).

180
 The other direction that if the Independent Set problem has an ap-
proximation algorithm of running time O(t(n)) and approximation ratio
r(n) then so does the Clique problem can be proved in a similar way.
Corollary 34.10 The Clique problem has no polynomial time approxima-
tion scheme unless P = NP.
proof. This follows directly from the combination of Theorem 34.9 and
Theorem 34.7.
According to Theorem 34.9, we can say that essentially there is no dif-
ference in the approximability between the Clique problem and the Inde-
pendent Set problem. In fact, a result can be obtained which is much
stronger than Theorem 34.7 and Corollary 34.10 on the approximability of
these two optimization problems. We will discuss this in the next lecture.

181
 CPSC-669 Computational Optimization
Lecture #35, November 17, 1995
Lecturer: Professor Jianer Chen
Scribe: Xiaotao Chen
Revision: Jianer Chen
35 Independent Set is not in APX
In this lecture, We present results on the approximability of the Indepen-
dent Set problem and the Clique problem that strengthen Theorem 34.7
and Corollary 34.10.
Let G1 = (V1; E1) and G2 = (V2; E2) be two graphs. The composition
graph G = G1 [G2] of these two graphs is the graph G = (V; E ) that has
vertex set V = V1  V2. Two vertices [u1 ; u2] and [v1; v2] in the graph G are
adjacent, where u1 and v1 are vertices in G1 and u2 and v2 are vertices in
G2, if and only if either (u1; v1) is an edge in G1, or u1 = v1 and (u2 ; v2) is
an edge in G2. A convenient way to view the composition graph G = G1 [G2]
is as being constructed by replacing each vertex of G1 by a copy of G2 and
then replacing each edge of G1 by a complete bipartite subgraph that joins
every vertex in the copy corresponding to one endpoint to every vertex in
the copy corresponding to the other endpoint.
Lemma 35.1 Let fu1; : : :; uk g be an independent set of k vertices in the
graph G1 and let fv1 ; : : :; vh g be an independent set of h vertices in the
graph G2, then the kh vertices
[ui ; vj ] 1  i  k and 1  j  h
form an independent set in the composition graph G1[G2].
proof. By the denition, no two vertices in fu1 ; : : :; uk g are adjacent in
the graph G1 and no two vertices in fv1; : : :; vh g are adjacent in the graph
G2. According to the denition of the composition graph G1 [G2], it is easy
to check that no two vertices [ui ; vj ] and [ui0 ; vj 0 ] in the list given in the
lemma are adjacent in the graph G1[G2].
Let G be any graph. We dene a graph dG by iterating the composition
operation. Inductively, 1G = G, and for d > 1, dG = (d 1)G[G]. Note
that if the graph G has n vertices, then the graph dG has nd vertices.
182
Lemma 35.2 Let d be a xed positive integer and K be an integer satisfying
K > (k 1)d. If the composition graph dG has an independent set Sd of K
vertices, then the graph G has an independent set S that contains at least k
vertices. Moreover, the independent set S of the graph G can be constructed
from the independent set Sd of the graph dG in polynomial time.
proof. First note that since d is a xed constant, the number of vertices
in the graph dG is nd , which is a polynomial of the number n of vertices in
the graph G. Therefore, the running time of any polynomial time algorithm
on the graph dG is also bounded by a polynomial of n.
We prove the lemma by induction on the integer d. The lemma is cer-
tainly true for the case d = 1.
Now consider the graph dG = (d 1)G[G]. Suppose that the independent
set Sd in the graph dG is
Sd = f[u1; v1]; [u2; v2]; : : :; [uK ; vK ]g
where ui 's are vertices in the graph (d 1)G and vj 's are vertices in the
graph G.
We partition the vertices in the set Sd into groups H1 , : : :, Hm such that
all vertices in each group Hj have the same rst coordinate. There are two
possible cases.
If one Hi of the groups contains at least k vertices:
Hi = f[ui ; w1]; [ui; w2]; : : :; [ui; wk0 ]g
where k0  k, then since no two of these k0 vertices are adjacent in the
graph dG = (d 1)G[G], by the denition, no two of the k0 vertices w1, : : :,
wk0 are adjacent in the graph G. That is, the set S = fw1; : : :; wk0 g is an
independent set of at least k vertices in the graph G. It is also easy to see
that if such a group Hi exists in the set Sd , then the set S = fw1; : : :; wk0 g
can be constructed in polynomial time.
If none of the groups H1, : : :, Hm in the set Sd contains more than k 1
vertices, then we have m(k 1)  K , which implies m  K=(k 1) >
(k 1)d 1 . Pick any vertex [ui ; wi] from the group Hi , for i = 1; : : :; m,
since no two of the vertices
[u1; w1]; [u2; w2]; : : :; [um; wm]
are adjacent in the graph dG, by the denition, no two of the vertices u1 ,
: : :, um are adjacent in the graph (d 1)G. Thus, we obtain an independent
183
set Sd 1 of m > (k 1)d 1 vertices in the graph (d 1)G:
Sd 1 = fu1; u2; : : :; umg
It is easy to see that the independent set Sd 1 of the graph (d 1)G can be
constructed in polynomial time from the independent set Sd of the graph
dG. Now by our inductive hypothesis, an independent set S of at least k
vertices in the graph G can be constructed in polynomial time from the
independent set Sd 1 of more than (k 1)d 1 vertices in the graph (d 1)G.
Because d is a xed constant, we conclude that an independent set S of at
least k vertices in the graph G can be constructed in polynomial time from
the independent set Sd of more than (k 1)d vertices in the graph dG.
This completes the proof of the lemma.
Theorem 35.3 The number of vertices in a maximum independent set of
the graph G is k if and only if the number of vertices in a maximum inde-
pendent set of the graph dG is kd .
proof. Suppose that a maximum independent set of the graph G contains
k vertices. Applying induction on Lemma 35.1, we can easily derive that
a maximum independent set of the graph dG has at least kd vertices. If a
maximum independent set of the graph dG contains more than kd vertices,
then by Lemma 35.2, the graph G would contain an independent set of
more than k vertices. This contradicts the assumption that a maximum
independent set in G has k vertices. Thus, a maximum independent set of
the graph dG contains exactly kd vertices.
Conversely, if a maximum independent set of the graph dG has kd ver-
tices, then by Lemma 35.2, a maximum independent set of the graph G
contains at least k vertices. A maximum independent set of G cannot con-
tain more than k vertices since otherwise, by Lemma 35.1, a maximum
independent set of dG would contain more than kd vertices.
Now we come back to the approximability of the Independent Set
problem and the Clique problem.
Lemma 35.4 If the Independent Set problem has a polynomial time ap-
proximation algorithm Ac with approximation ratio bounded by a xed con-
stant c  1, then the Independent Set problem has a polynomial time
approximation scheme.

184
proof. We construct a polynomial time approximation scheme for the
Independent Set problem from the polynomial time approximation al-
gorithm Ac of constant approximation ratio for the problem. Consider the
following algorithm for the Independent Set problem.
Algorithm 35.1 PTAS-IS
Input: a graph G
and a constant >0
Output: an independent set in S G
1. d = d(c 1)=e;
2. construct the composition graph ; dG
3. apply algorithm Ac to construct an independent set Sd
in the graph ;dG
4. construct an independent set of S k
vertices in the
graph G S
from the set d , where k
is the largest
integer such that d ( jS j > k
1)d .
Since d  (c 1)=, we have (1 + )d  1 + d  c. Note that when c and
 are xed constants, d is also a xed constant. Thus, if the graph G has n
vertices, then the number of vertices of the composition graph dG is bounded
by a polynomial of n, and the composition graph dG can be constructed in
polynomial time. Consequently, the independent set Sd in the graph dG can
be constructed in time polynomial in n. Finally, according to Lemma 35.2,
the independent set S in the graph G can be constructed from Sd in time
polynomial in n. In conclusion, the algorithm PTAS-IS is a polynomial time
algorithm.
Let Opt(G) be the number of vertices in a maximum independent set of
the graph G. By Theorem 35.3, the number Opt(dG) of vertices in a max-
imum independent set of the graph dG is (Opt(G))d. By the construction
we have jSd j  kd = jS jd. Finally, since the independent set Sd of the graph
dG is produced by the algorithm Ac , by our assumption, Opt(dG)=jSdj  c.
With all these relations, we obtain
 Opt(G) d (Opt(G))d Opt(dG)
jS j = jS jd  jS j  c  (1 + )d
d
which gives
Opt(G)  1 + 
jS j

185
Therefore, for any xed constant  > 0, the algorithm PTAS-IS constructs in
polynomial time an independent set S for the graph G such that the approx-
imation ratio Opt(G)=jS j is bounded by 1 + . That is, the Independent
Set problem has a polynomial time approximation scheme.
Since we already know that the Independent Set problem has no
polynomial time approximation scheme unless P = NP (Theorem 34.7),
Lemma 35.4 derives the following theorem that is stronger than Theo-
rem 34.7.
Theorem 35.5 Unless P = NP, the Independent Set problem is not in
the class APX. In other words, unless P = NP, the Independent Set prob-
lem has no polynomial time approximation algorithm whose approximation
ratio is bounded by a xed constant c.
Corollary 35.6 The Clique problem has no polynomial time approxima-
tion algorithm whose approximation ratio is bounded by a xed constant c,
unless P = NP.
proof. This follows from Theorem 34.9 and Theorem 35.5.

186
 CPSC-669 Computational Optimization
Lecture #36, November 20, 1995
Lecturer: Professor Jianer Chen
Scribe: Xiaotao Chen
Revision: Jianer Chen
36 Vertex Cover has no PTAS
After the establishment of the fundamental theorem (Theorem 33.3) that
claims the non-approximability of the Max-3Sat problem, we have seen
that the non-approximability of other optimization problems, such as the
Max-Sat problem and the Independent Set problem, can be derived by
reducing the Max-3Sat problem to them. The reductions used for these
problems are essentially the same as the ones that are used in the proofs for
the NP-completeness of the corresponding decision versions of the problems.
Since there has been a long line of reductions that reduce one NP-complete
problem to another, we would wonder whether we can somehow simply
modify these reductions for decision problems to the world of optimization
problems so that the non-approximability of all optimization problems can
be derived.
Researchers quickly realized the dierence in these two worlds. To il-
lustrate the dierence, we use the Vertex Cover as an example. Recall
that given a graph G, a subset S of vertices of G forms a vertex cover of G
if every edge in G has at least one end in the set S . The Vertex Cover
problem is formulated as follows.
Vertex Cover
Input: a graph G
Output: a vertex cover S of G with jS j minimized
Lemma 36.1 Let G = (V; E ) be a graph. A set S  V of vertices is a
vertex cover of the graph G if and only if the set V S is an independent
set in the graph G. In particular, S is a minimum vertex cover of the graph
G if and only if V S is a maximum independent set in the graph G.
proof. If S is a vertex cover of the graph G, then every edge in G has at

187
least one end in S . Therefore, no edge in G has both ends in V S . That
is, the set V S is an independent set in the graph G.
If V S is an independent set of the graph G, then no two vertices in
V S are adjacent. That is, no edge has both ends in the set V S , or
equivalently, every edge has at least one end in S . Thus, the set S is a vertex
cover.
The rest of the lemma follows directly.
Therefore, suppose that we know that the decision version of the Inde-
pendent Set problem \given a graph G and k, does G have an independent
set of k vertices?" is NP-complete, then we can use Lemma 36.1 to show
the NP-completeness of the decision version of the Vertex Cover prob-
lem \given a graph G and k, does G have a vertex cover of k vertices?"
by reducing the decision version of the Independent Set problem to it:
asking if a given graph G has an independent set of k vertices is equivalent
to asking if the graph G has a vertex cover of n k vertices. Therefore, the
problem of nding an independent set of k vertices in a graph is not harder
than the problem of nding a vertex cover of n k vertices in the same
graph. Thus, the hardness of the rst problem implies the hardness of the
second problem.
However, the above reduction would not work if we approximate the op-
timal solutions in the problems. For instance, let G = (V; E ) be a graph of
1000 vertices in which a maximum independent set has 10 vertices while a
minimum vertex cover contains 990 vertices. Now suppose we want to derive
an approximation for the maximum independent set from an approximation
for the minimum vertex cover. Let S be a vertex cover of 950 vertices in G.
Then S is a pretty good approximation for the minimum vertex cover (with
approximation ratio 990=950 < 1:05). However, if we use the above reduc-
tion to get the independent set V S for the graphG, we obtain a very bad
approximation V S for the maximum independent set (with approximation
ratio 50=10 = 5). Therefore, even Lemma 36.1 suggests a reduction that re-
duces the Independent Set problem to the Vertex Cover problem, the
reduction does not preserve the approximation ratio when we apply approx-
imation algorithms. The hardness of approximability of the Independent
Set thus does not imply the hardness of approximability of the Vertex
Cover problem. In fact, as we have shown, the Independent Set has no
polynomial time approximation algorithm with a constant approximation ra-
tio unless P = NP (Theorem 35.5) while the Vertex Cover problem has
a simple approximation algorithm of approximation ratio 2 (Theorem 29.1).

188
 Therefore, to study approximability of optimization problems, we need to
consider reductions that somehow preserve the approximability. Before we
present the formal denition for the reduction, we rst introduce a notation.
Denition 36.1 Let Q = hI; S; f; opti be an optimization problem. Let
x 2 I be an instance of Q and y 2 S (x) be a solution to the instance x. The
relative error EQ (x; y ) of the solution y is dened by
8
< Opt(x)
f (x;y) 1 if opt = max
EQ (x; y) = : f (x;y)
Opt(x) 1 if opt = min
Simply speaking, the relative error EQ(x; y ) of the solution y is the approx-
imation ratio of y minus 1. Note that the relative error EQ(x; y ) is always
a non-negative number.
Now we are ready for giving the denition of our reduction that preserves
the approximation ratio.
Denition 36.2 An optimization problem Q1 = hI1; S1; f1; opt1i can be
E-reducible to an optimization problem Q2 = hI2 ; S2; f2 ; opt2i, in written
Q1 E Q2 , if there are two polynominal time computable functions g (
) and
h(
;
), a polynomial p(
) and a constant , such that
1. for any x1 2 I1 , g (x1) = x2 2 I2 , satisfying Opt2 (x2)  p(jx1j)Opt1 (x1)
and Opt1 (x1)  p(jx1j)Opt2(x2);
2. for any y2 2 S2(x2 ), h(x1 ; y2) = y1 2 S1(x1 ) such that
E1 (x1; y1)  E2(x2; y2)
where Opt1 (x1 ) and Opt2 (x2) are the optimal values for the instances x1 and
x2 of the problems Q1 and Q2 , respectively, and E1 and E2 are the relative
errors for the problems Q1 and Q2 , respectively.
The denition of the E-reduction seems very natural from the viewpoint
of polynomial time approximability of optimization problems. Condition
1 and the polynomial time computability of the functions g and h ensure
that the reduction is a polynomial time transformation, while condition 2
requires that the transformation preserves the approximability. The only
thing that looks a bit less natural is the requirement that the optimal values
189
of the instances are related by a polynomial factor. We will see that this
requirement makes the E-reduction the canonical reduction for an important
class of optimization problems. Moreover, in most cases, this requirement is
naturally satised. For example, if the optimal values of both optimization
problems Q1 and Q2 are bounded by a polynomial of their input instance
length, then this requirement is automatically satised.
The E-reduction from the optimization problem Q1 to the optimization
problem Q2 provides a systematic technique for designing approximation
algorithms for the problem Q1 based on approximation algorithms for the
problem Q2 , as shown by the following lemmas.
Lemma 36.2 Let Q1 and Q2 be two optimization problems. If Q1 E Q2
and Q2 has a fully polynomial time approximation scheme, then so does Q1 .
proof. Let Q1 = hI1; S1; f1; Opt1i and Q2 = hI2 ; S2; f2 ; Opt2i. Let A2 be
a fully polynomial time approximation scheme for the problem Q2. Suppose
that the reduction Q1 E Q2 is given as stated in Denition 36.2.
We design an approximation algorithm for the problem Q1 as follows.
For any constant  > 0, given x1 2 I1, we (1) compute x2 = g (x1) 2 I2 ; (2)
apply the algorithm A2 for Q2 on x2 to get a solution y2 for x2 satisfying
E2(x2; y2)  =; and nally (3) construct the solution y1 = h(x1 ; y2) for
the instance x1 of Q1 .
Since A2 is a fully polynomial time approximation scheme for Q2 , the
solution y2 can be constructed in time polynomial in jx2 j and =, thus in
time polynomial in jx1 j and 1=. The other steps of the above algorithm
take time polynomial in jx1 j. Therefore, the total running time of the above
algorithm is bounded by a polynomial of jx1 j and 1=.
Since the reduction is an E-reduction, we have
E1(x1 ; y1)  E2(x2; y2)  
Thus, the above algorithm has approximation ratio 1 + , thus, is a fully
polynomial time approximation scheme for the problem Q1 .
In a similar way, we can prove
Lemma 36.3 Let Q1 and Q2 be two optimization problems. If Q1 E Q2
and Q2 has a polynomial time approximation scheme, then so does Q1.

190
Lemma 36.4 Let Q1 and Q2 be two optimization problems. If Q1 E Q2
and Q2 is in the class APX (i.e., Q2 has a polynomial time approximation
algorithm whose approximation ratio is bounded by a xed constant), then
so is Q1 .
In particular, Lemma 36.3 implies
Lemma 36.5 Let Q1 and Q2 be two optimization problems and Q1 E Q2.
If Q1 does not have a polynomial time approximation scheme unless P =
NP, then Q2 does not have a polynomial time approximation scheme unless
P = NP.
Example 36.3 We give an example to illustrate these ideas. In Lecture
34, we presented a reduction from Q1 , the Max-3Sat problem, to Q2, the
Independent Set problem as follows. Given an instance x1 of the Max-
3Sat problem, where x1 is a set of clauses of at most 3 literals (x1 was
written as E in the discussion of Lecture 34), we construct a graph x2 that
is an instance for the Independent Set problem (x2 was written as GE
in the discussion of Lecture 34). This corresponds to the polynomial time
computable function g (
): g (x1) = x2. The condition that the optimal
values of x1 and x2 are related by a polynomial factor is automatically
satised since both of x1 and x2 have their optimal values bounded by a
polynomial of their input instance length. Now for any solution y2 to x2 ,
where y2 is an independent set in the graph x2 (y2 was written as S in the
discussion of Lecture 34), a truth assignment y1 to the clauses in x1 can be
constructed in polynomial time (y1 was written as S in the discussion of
Lecture 34). This corresponds to the polynomial time computable function
h(
;
): y1 = h(x1; y2). Since we have Opt1 (x1) = Opt2 (x2) (Corollary 34.5),
f1 (x1 ; y1)  f2(x2; y2 ) (Lemma 34.4), we eventually have
E1 (x1; y1) = fOpt 1 (x1) 1  Opt2 (x2 ) 1 = E (x ; y )
2 2 2
1 (x1; y1) f2 (x2; y2)
Thus, if we let  = 1, then the above reduction is an E-reduction from the
Max-3Sat problem to the Independent Set problem. Since the Max-
3Sat problem has no polynomial time approximation scheme unless P = NP
(Theorem 33.3), by Lemma 36.5, we derive directly that the Independent
Set problem has no polynomial time approximation scheme unless P = NP.
Now we apply the E-reduction to show the non-approximability of the
Vertex Cover problem. As we explained at the beginning of this lecture,

191
the reduction from the Independent Set problem to the Vertex Cover
problem does not seem to preserve approximation ratio. Thus, a reduction
from another problem seems more proper. We present an E-reduction from
the Max-3Sat problem to the Vertex Cover problem.
The reduction from the Max-3Sat problem to the Vertex Cover
problem is the same as the one that reduces the Max-3Sat problem to the
Independent Set problem.
Given an instance E = fC1; C2; : : :; Cmg of the Max-3Sat problem,
where each Ci is a clause of at most 3 literals in fx1; : : :; xn g. The graph
GE = (VE ; AE ) is constructed as follows.
Every literal occurrence l in a clause Ci in E induces a vertex in the
graph GE , which will be named by l(i). For any pair of vertices l1(i) and l2(j )
in GE , there is an edge connecting them if and only if either
1. i = j , i.e., the literals l1(i) and l2(j ) belong to the same clause in E ; or
2. i 6= j and l1(i) = l2(j ), i.e., the literals l1(i) and l2(j ) belong to dierent
clauses in E and they negate each other.
This completes the transformation from the instance E of the Max-
3Sat problem to an instance GE of the Vertex Cover problem. Note
that again the condition that the optimal values of E and GE are related
by a polynomial factor is automatically satised since both E and GE have
their optimal value bounded by their input length.
Now we show how the transformation from a solution S for the instance
GE , where S is a vertex cover of GE , to a solution S for the instance E ,
where S is an assignment to fx1; : : :; xn g, is constructed.
Given a vertex cover S of the graph GE = (VE ; AE ), we rst construct
the independent set VE S in the graph GE . Then based on the indepen-
dent set VE S , we apply Lemma 34.4 to construct an assignment 0 to the
variables fx1 ; : : :; xn g such that 0 satises at least jVE S j clauses in E . If
the assignment 0 satises at least m=2 clauses in E , then we let S = 0 .
If the assignment 0 satises less than m=2 clauses in E , then we apply Al-
gorithm 31.1 ApprxMaxSat to construct the assignment S that satises at
least m=2 clauses in E (see Lemma 31.2), ignoring the assignment 0 . There-
fore, the assignment S always satises at least maxfm=2; jVE S jg clauses
in E . According to Lemma 34.4, the assignment 0 can be constructed from
the independent set VE S in polynomial time. Moreover, Algorithm 31.1
ApprxMaxSat runs in polynomial time. We conclude that the assignment S

192
to fx1; : : :; xn g can be constructed from the vertex cover S in polynomial
time. This completes the transformation from a solution S for the instance
GE of the Vertex Cover problem to a solution S for the instance E of
the Max-3Sat problem.
Now we analyze the relative errors of the solutions S and S .
Lemma 36.6 A minimum vertex cover of the graph GE = (VE ; AE ) con-
tains at most 5jVE j=6 vertices.
proof. Each clause of the set E contains at most 3 literals, and there
are m clauses in the set E . Since there is a one-to-one correspondence
between the literal occurrences in E and the vertices in the graph GE , the
number jVE j of vertices of the graph GE is bounded by 3m, which gives
m=2  jVE j=6.
By Algorithm 31.1 and Lemma 31.2, we know that there is an assignment
to fx1; : : :; xn g that satises at least m=2 clauses in E . By Corollary 34.5, a
maximum independent set in the graph GE contains at least m=2 vertices.
Now by Lemma 36.1, a minimum vertex cover of the graph GE contains at
most
jVE j m=2  jVE j jVE j=6 = 5jVE j=6
vertices. The lemma is proved.
Let Optsat (E ) be the optimal value for the instance E of the Max-3Sat
problem, let Optis (GE ) be the optimal value for the instance GE of the
Independent Set problem, and let Optvc (GE ) be the optimal value for the
instance GE of the Vertex Cover problem. According to Corollary 34.5
and Lemma 36.1
Optsat(E ) = Optis (GE ) = jVE j Optvc (GE ) (14)
Let Apxsat (S ) be the number of clauses in E that are satised by the
assignment S . By the construction of the assignment S , we have
Apxsat(E )  maxfm=2; jVE S jg
Let Esat (E; S ) be the relative error of the solution S to the instance E
of the Max-3Sat problem and let Evc (GE ; S ) be the relative error of the
solution S to the instance GE of the Vertex Cover problem, we have
Optsat(E ) 1 and E (G ; S ) = jS j
Esat(E; S ) = Apx
sat (S )
vc E Optvc (GE ) 1
193
Now from
Optsat (E ) 1 = Optsat (E ) Apxsat (S )
Esat(E; S ) = Apx
sat (S ) Apxsat (S )
we combine the relation Apxsat (S )  maxfm=2; jVE S jg with Equation
(14) and obtain
) jVE S j = jS j (jVE j Optsat (E ))
Esat(E; S )  Optsat(Em= 2 m=2
= jS j (jVE jm=Opt
2
is (GE )) = jS j Optvc (GE )
m=2
Now by Lemma 36.6 Optvc (GE )  5jVE j=6 and note jVE j  3m, we get
Esat(E; S )  jSOpt
j Optvc (GE )
 jS j  vc (GE )=5
= 5 Opt (G ) 1 = 5Evc (GE ; S )
vc E
We summarize this analysis in the following lemma.
Lemma 36.7 The reduction constructed above that reduces the Max-3Sat
problem to the Vertex Cover problem is an E-reduction.
Combining Lemma 36.7 with Theorem 33.3 and Lemma 36.5, we obtain
the following result for the Vertex Cover problem.
Theorem 36.8 The Max-3Sat problem is E-reducible to the Vertex Cover
problem. In consequence, the Vertex Cover problem has no polynomial
time approximation scheme unless P = NP.

194
 CPSC-669 Computational Optimization
Lecture #37, November 27, 1995
Lecturer: Professor Jianer Chen
Scribe: Xiaotao Chen
Revision: Jianer Chen
37 The E-reducibility and the class APX-PB
The E-reduction introduced in the last lecture plus Theorem 33.3 have led
to signicant progress recently in the study of the approximability of opti-
mization problems. Here we mention some of the recent results related to
this direction.
Denition 37.1 An optimization problem Q = hI; S; f; opti is in the class
APX-PB if Q is in the class APX and there is a polynomial p(
) such that
for all instances x 2 I , we have Opt(x)  p(jxj). Here the letters PB stand
for \polynomially bounded".
Not all optimization problems in APX are in APX-PB. For example, the
Knapsack problem, the Multi-Processor Scheduling problem, and
the
-Traveling Salesman problem are all in the class APX but none
of them is in the class APX-PB. On the other hand, a very large class
of important optimization problems are in the class APX-PB, such as the
Max-Sat problem, Vertex Cover problem, 3-D Matching problem, and
Bin Packing problem. By Theorem 17.1, we know that an optimization
problem in APX-PB has no fully polynomial time approximation scheme
unless the problem can be solved in polynomial time. Therefore, for an NP-
hard optimization problem in the class APX-PB, the most interesting thing
is to decide whether it admits a polynomial time approximation scheme.
Since the E-reducibility requires that the optimal values of the instances
in the original problem and in the transformed problem be related by a
polynomial factor, it is impossible to reduce an optimization problem in
APX but not in APX-PB to an optimization problem in APX-PB. In fact,
this requirement in the E-reducibility makes the class APX-PB closed under
the E-reducibility, as shown in the following lemma.
Lemma 37.1 Let Q1 and Q2 be two optimization problems. If Q1 E Q2
and Q2 is in the class APX-PB, then so is Q1 .

195
proof. Let Q1 = hI1; S1; f1; opt1i and Q2 = hI2 ; S2; f2 ; opt2i. By
Lemma 36.4, since the optimization problem Q2 is in the class APX, the
optimization problem Q1 is also in the class APX. Moreover, for any x1 2 I1 ,
since the E-reduction transforms x1 into an instance x2 2 I2 in polynomial
time, we have jx2 j bounded by a polynomial of jx1j. Now since Q2 is in the
class APX-PB, the value Opt2 (x2) is bounded by a polynomial of jx2j thus
by a polynomial of jx1 j. Now from the denition of the E-reducibility, the
optimal value Opt1 (x1) of the instance x1 is bounded by Opt2 (x2) times a
polynomial of jx1 j. We conclude that Opt1 (x1) is bounded by a polynomial
of jx1j. That is, the problem Q1 is in the class APX-PB.
The following important result, which was a little unexpected, has been
derived recently. The proof of the theorem is omitted. Interested students
can talk to the instructor for a discussion of the proof.
Theorem 37.2 An optimization problem Q is in the class APX-PB if and
only if it is E-reducible to the Max-3Sat problem.
Remark 37.2 Theorem 37.2 is derived from a modication of Theorem 33.1.
Theorem 37.2 is a very powerful theorem. For example, our fundamental
theorem, Theorem 33.3, can be derived directly from Theorem 37.2 without
using Theorem 33.1. We give a complete proof for this.
According to Algorithm 23.1 First-Fit and Theorem 23.1, the Bin
Packing problem is in the class APX-PB. By Theorem 37.2, the Bin
Packing problem is E-reducible to the Max-3Sat problem. Now if the
Max-3Sat problem has a polynomial time approximation scheme, then by
Lemma 36.3, the Bin Packing problem has a polynomial time approxi-
mation scheme. However, by Theorem 23.2, there is no polynomial time
approximation algorithm of approximation ratio less than 1:5 for the Bin
Packing problem unless P = NP. In particular, the Bin Packing prob-
lem has no polynomial time approximation scheme unless P = NP. This
proves that the existence of a polynomial time approximation scheme for
the Max-3Sat problem implies P = NP. This is Theorem 33.3.
Remark 37.3 According to Lemma 37.1, the E-reduction cannot transform
an optimization problem in APX but not in APX-PB to an optimization
problem in APX-PB. Thus, the class APX-PB in Theorem 37.2 cannot be
replaced by the class APX since the Max-3Sat problem is in the class APX-
PB. However, if the E-reduction is replaced by another reduction, called the
196
PTAS-reduction, then Theorem 37.2 is also true for the class APX, that is, an
optimization problem is in the class APX if and only if it is PTAS-reducible
to the Max-3Sat problem. The denition of the PTAS-reducibility is a
bit more technical, but it still preserves PTAS approximability and APX
approximability. More specically, suppose that a problem Q1 is PTAS-
reducible to a problem Q2. Then if Q2 has a polynomial time approximation
scheme then so does Q1 , and if Q2 is in the class APX then so is Q1 . I am
not going to give a detailed discussion along this line. Instead, I refer the
interested students to the related literature.
Theorem 37.2 motivates the following denition.
Denition 37.4 An optimization problem Q is ApxPB-hard if every opti-
mization problem in the class APX-PB is E-reducible to Q. An optimization
problem Q is ApxPB-complete if Q is in the class APX-PB and Q is ApxPB-
hard.
According to the denition, the Max-3Sat problem is ApxPB-complete.
Theorem 37.3 An ApxPB-hard optimization problem Q has no polynomial
time approximation scheme unless P = NP.
proof. If the problem Q is ApxPB-hard, then by the denition, the
Max-3Sat problem can be E-reducible to the problem Q. Now the theorem
follows directly from Lemma 36.5 and Theorem 33.3.
Thus, the ApxPB-hardness implies the diculty of approximation of
an optimization problem. This provides a systematic technique for deriv-
ing non-approximability for optimization problems. To derive the ApxPB-
hardness for optimization problems, we need the following lemma.
Lemma 37.4 If Q1 E Q2 and Q2 E Q3, then Q1 E Q3.
proof. Let Q1 = hI1; S1; f1; opt1i, Q2 = hI2; S2; f2; opt2i, and Q3 =
hI3; S3; f3; opt3i. Let g1(
) and h1(
;
) be the functions, p1(
) be the poly-
nomial, and 1 be the constant that constitute the E-reduction from Q1 to
Q2 , and let g2 (
) and h2(
;
) be the functions, p2 (
) be the polynomial, and
2 be the constant that constitute the E-reduction from Q2 to Q3. Without
loss of generality, suppose that both functions g1(
) and g2 (
) are computable

197
in time p(n), where p(
) is a polynomial. Then it is easy to check that the
functions
g(x1) = g2(g1(x1)) and h(x1; y3) = h1(x1 ; h2(g1(x1); y3))
the polynomial
p(n) = p1(n)
p2(p(n))
and the constant
 = 1 2
constitute an E-reduction from the optimization problem Q1 to the opti-
mization problem Q3 .
Lemma 37.4 immediately gives
Lemma 37.5 If an optimization problem Q1 is ApxPB-hard, and Q1 E
Q2 , then the optimization problem Q2 is ApxPB-hard.
Now our previous study gives the following theorem.
Theorem 37.6 The Max-3Sat problem, the Max-Sat problem, and the
Vertex Cover problem are ApxPB-complete. The Independent Set
problem and the Clique problem are ApxPB-hard.
proof. Algorithm 31.1 ApprxMaxSat shows that the Max-3Sat problem
and the Max-Sat problem are in the class APX-PB. Theorem 37.2 shows
that the Max-3Sat problem is ApxPB-hard. The proof of Lemma 34.1
shows the ApxPB-hardness for the Max-Sat problem. Theorem 36.8 gives
the ApxPB-hardness of the Vertex Cover problem, and Theorem 29.1
shows that the Vertex Cover problem is in the class APX-PB.
Finally, Example 36.3 shows the ApxPB-hardness for the Independent
Set problem, and Theorem 34.9 gives the ApxPB-hardness for the Clique
problem.
There are a number of more restricted optimization problems that are
also ApxPB-complete. Note that the ApxPB-hardness of more restricted
optimization problems sometimes is more useful in our derivation of non-
approximability for our own optimization problems: our own problem may
not be strong enough to have a simple E-reduction from an unrestricted
ApxPB-hard optimization problem. On the other hand, a restricted version
198
of the problem may look more similar to our own problem and an E-reduction
may be readily available.
Let d be a xed positive integer. We dene the following restricted
optimization problems.
d-Occurrence Max-3Sat
Input: a set E of clauses C1; C2; : : :; Cm on fx1; : : :; xn g such
that each clause has at most 3 literals and each variable xi ap-
pears, either as xi or as xi , at most d times in E
Output: a truth assignment on fx1; : : :; xng that satises the
maximum number of the clauses in E
d-Occurrence Max-2Sat
Input: a set E of clauses C1; C2; : : :; Cm on fx1; : : :; xn g such
that each clause has at most 2 literals and each variable xi ap-
pears, either as xi or as xi , at most d times in E
Output: a truth assignment on fx1; : : :; xng that satises the
maximum number of the clauses in E
d-Degree Vertex Cover
Input: a graph G in which each vertex has degree at most d
Output: a vertex cover S of G with jS j minimized
d-Degree Independent Set
Input: a graph G in which each vertex has degree at most d
Output: an independent set S of G with jS j maximized
Theorem 37.7 The 3-Occurrence Max-3Sat problem is ApxPB-complete.
proof. The 3-Occurrence Max-3Sat problem is a restricted version
of the Max-Sat problem, which is in the class APX-PB (Theorem 37.6).
Thus, the 3-Occurrence Max-3Sat problem is in the class APX-PB.
The proof that the 3-Occurrence Max-3Sat problem is ApxPB-hard
is more complicated, which uses the techniques called Amplier and Ex-
pander. We omit the detailed proof here. Interested students are encouraged
to talk to the instructor.

199
Theorem 37.8 The 4-Degree Vertex Cover problem and the 4-Degree
Independent Set problem are ApxPB-complete.
proof. We rst consider the 4-Degree Independent Set problem.
It is a bit surprising that the 4-Degree Independent Set problem is
in the class APX-PB. As we have seen, the unrestricted Independent Set
problem is not in the class APX unless P = NP (Theorem 35.5). However,
when the degree of the vertices of a graph G is bounded by a xed constant
c, an independent set of the graph G with approximation ratio bounded by
c +1 can be constructed easily by the following process: start with an empty
set S . Each time we pick one vertex v from the graph G and put it in S ,
and delete all neighbors of v in the graph G. We iterate this until there is
no vertex in the graph G. It is easy to see that the set S constructed by this
process forms an independent set in the graph G. Since the degree of the
vertices of the graph G is bounded by c, each time we add a vertex v to the
set S , we delete at most c + 1 vertices from the graph (including the vertex
v ). Therefore, the set S contains at least n=(c + 1) vertices, where n is the
number of vertices in the graph G. Since an independent set of the graph G
contains at most n vertices, the independent set S has approximation ratio
at most n=(n=(c + 1)) = c + 1. In particular, when c = 4, we have that the
4-Degree Independent Set problem is in the class APX-PB.
To show the ApxPB-hardness of the 4-Degree Independent Set prob-
lem, we E-reduce the 3-Occurrence Max-3Sat problem to it. The re-
duction is exactly the same as the one we used to reduce the Max-3Sat
problem to the Independent Set problem (see Lecture Notes 34). That
is, given a set E of clauses, we construct a graph GE such that each literal
occurrence in E corresponds to a vertex in GE , and two vertices in GE are
adjacent if and only if either they are in the same clause in E or they negate
each other. Note that if E is an instance of the 3-Occurrence Max-3Sat
problem, then each vertex l in GE has at most 4 neighbors | two of them
correspond to the literals in the same clause, and two of them correspond
to the other occurrences of l in E . Therefore, the graph GE is an instance
of the 4-Degree Independent Set problem. As we have shown in Lec-
ture 34, this reduction is an E-reduction. We conclude that the 4-Degree
Independent Set problem is ApxPB-complete.
The problem 4-Degree Vertex Cover problem is in the class APX-
PB because the unrestricted version of the Vertex Cover problem is in the
class APX-PB. As we have seen in Lecture Notes 36, the E-reduction from
the Max-3Sat problem to the Independent Set problem can be modied
200
to an E-reduction from the Max-3Sat problem to the Vertex Cover
problem. In particular, if E is an instance of the 3-Occurrence Max-
3Sat problem, then the corresponding graph GE is an instance of the 4-
Degree Vertex Cover problem. Thus, the 3-Occurrence Max-3Sat
problem is E-reducible to the 4-Degree Vertex Cover problem, which
gives the ApxPB-hardness of the 4-Degree Vertex Cover problem.
Finally, we consider the 5-Occurrence Max-2Sat problem. We rst
present an E-reduction from the 4-Degree Independent Set problem to
the 5-Occurrence Max-2Sat problem.
Let G = (V; E ) be a graph in which each vertex has degree at most 4,
where V = fv1 ; : : :; vn g, and E = fe1 ; : : :; emg. We construct an instance
SG for the 5-Occurrence Max-2Sat problem as follows. The boolean
variable set of SG is fv1 ; : : :; vn g. For each vertex vi in G, the set SG has a
1-literal clause (vi ), and for each edge eh = [vi ; vj ] in G, SG has a 2-literal
clause (vi _ vj ). Note that each vertex of G appears in one 1-literal clause
and in at most four 2-literal clauses. Thus, the set SG is an instance of the
5-Occurrence Max-2Sat problem. This completes the construction of
the instance SG for the 5-Occurrence Max-2Sat problem.
We call an assignment to fv1 ; : : :; vn g a setting assignment if it satises
all 2-literal clauses in the set SG .
Lemma 37.9 Let  be an assignment to the boolean variables fv1; : : :; vng.
Then there is a setting assignment 0 that satises at least as many clauses
in SG as  does. Moreover, the setting assignment 0 can be constructed
from the assignment  in polynomial time.
proof. If  is a setting assignment, then simply let 0 be . Otherwise,
suppose that the clause (vi _ vj ) is not satised by the assignment , then
 sets vi = 1 and vj = 1. Now we set vi = 0. Note this makes false the
1-literal clause (vi ), which was set true by , but makes true the 2-literal
clause (vi _ vj ), which was set false by . No other 1-literal clauses in SG are
turned from true to false. Moreover, since only the negation of vi appears
in 2-literal clauses in SG , no 2-literal clauses in SG are turned from true
to false. Therefore, setting vi = 0 will not decrease the number of clauses
satised by the assignment. Now we repeat the above process on each 2-
literal clause that is not satised by the assignment. Eventually, we obtain
a setting assignment 0 that satises at least as many clauses in SG as the
assignment  does.

201
Lemma 37.10 Let  be a setting assignment to SG and let (u1), : : :, (uk )
be the 1-literal clauses satised by the assignment , then fu1 ; : : :; uk g is an
independent set in the graph G.
proof. Consider any pair ui and uj . If the vertices ui and uj are adjacent
in G, then we have a 2-literal clause (ui _ uj ) in SG . Since  sets both ui
and uj true, the clause (ui _ uj ) is not satised by the assignment . This
contradicts the assumption that  is a setting assignment.
Let Optis (G) be the optimal value of the instance G for the 4-Degree
Independent Set problem, and let Optsat (SG ) be the optimal value of the
instance SG for the 5-Occurrence Max-2Sat problem.
Lemma 37.11 Let m be the number of edges in the graph G, then
Optis (G) + m = Optsat (SG )

proof. Let D = fu1; : : :; uk g be a maximum independent set in the

graph G. Consider the assignment D to fv1; : : :; vn g that sets uj = 1, for
j = 1; : : :; k, and sets all other variables 0. Thus, the assignment D satises
exactly k 1-literal clauses in SG . For each 2-literal clause (vi _ vj ), which
corresponds to an edge [vi ; vj ], since at least one of vi and vj is not in D,
the assignment D sets (vi _ vj ) true. That is, the assignment D satises
all 2-literal clauses. In conclusion, the assignment D satises k + m clauses
in SG . This gives
Optis (G) + m  Optsat (SG)
Now let  be an assignment to fv1 ; : : :; vn g that satises the largest
number of clauses in the set SG . By Lemma 37.9, we can assume that
the assignment  is a setting assignment. Let (u1 ), : : :, (uk ) be the 1-
literal clauses satised by . By Lemma 37.10, the set fu1 ; : : :; uk g is an
independent set in the graph G. Since the assignment  satises all m
2-literal clauses in SG , we get
Optis (G) + m  Optsat (SG)
This completes the proof.
Now we are ready for an E-reduction from the the 4-Degree Indepen-
dent Set problem to the 5-Occurrence Max-2Sat problem.

202
Lemma 37.12 The 4-Degree Independent Set problem is E-reducible
to the 5-Occurrence Max-2Sat problem.
proof. Given an instance G of the 4-Degree Independent Set prob-
lem, we use the transformation described above to construct an instance SG
for the 5-Occurrence Max-2Sat problem. The instance SG can certainly
be constructed from the instance G in polynomial time.
Now suppose that  is a solution to the instance SG , i.e.,  is assignment
to the boolean variables fv1 ; : : :; vn g in SG . We construct a solution D to
the instance G, where D is an independent set in the graph G, as follows.
We rst construct a setting assignment 0 that satises at least as many
clauses as  does. According to Lemma 37.9, the assignment 0 can be
constructed from the assignment  in polynomial time. Let (v1), : : :, (vk ) be
the 1-literal clauses satised by 0. Then by Lemma 37.10, D = fv1; : : :; vk g
is an independent set in the graph G. Now if jDj  n=5, we let D = D,
otherwise, we let D be an independent set of at least n=5 vertices in G.
Note that by the proof of Theorem 37.8, an independent set of at least n=5
vertices in G can be constructed in polynomial time when the degree of
vertices in the graph G is bounded by 4. This completes the construction
of the transformation that transforms the solution  of SG , which is an
instance of the 5-Occurrence Max-2Sat problem, to a solution D of G,
which is an instance of the 4-Degree Independent Set problem. By the
above discussion, the solution D can be constructed from the solution  in
polynomial time.
Now we analyze the relative errors. Let Apx() be the number of clauses
satised by the assignment . By the construction of the independent set
D, we have
jDj  maxfApx() m; n=5g (15)
Since each vertex of G has degree at most 4, the number m of edges in the
graph G, which also equals the number of 2-literal clauses in SG , is bounded
by 2n. Therefore,
Apx()  n + m  3n (16)
Let Eis (G; D) be the relative error of the solution D to the instance
G of the 4-Degree Independent Set problem, and Esat (SG ; ) be the
relative error of the solution  for the instance SG of the 5-Occurrence
Max-2Sat problem. Using Lemma 37.11, together with Equations (15) and

203
(16), We have
Eis(G; D) = OptjDis (jG) 1 = Optis (jGD) j jDj
 
 Opt is ( G ) ( Apx (  ) m ) = Optis (G) + m Apx()
n=5 n=5
= Optsat (SGn=) 5 Apx()  OptsatApx
(SG ) Apx()
()=15
= 15Esat(SG ; )
This completes the proof that the above reduction from the 4-Degree In-
dependent Set problem to the 5-Occurrence Max-2Sat problem is an
E-reduction.
Theorem 37.13 the Max-2Sat problem and the 5-Occurrence Max-
2Sat problem are ApxPB-complete.
proof. It is easy to see that both of these two problems are in the class
APX-PB.
The ApxPB-hardness of the 5-Occurrence Max-2Sat problem is de-
rived from Lemma 37.12
Finally, since the 5-Occurrence Max-2Sat problem is a restricted
version of the Max-2Sat problem, we conclude that the Max-2Sat problem
is also ApxPB-hard.
We will study more ApxPB-complete optimization problems in the rest
of this course.

204
 CPSC-669 Computational Optimization
Lecture #38, November 29, 1995
Lecturer: Professor Jianer Chen
Scribe: Balarama Varanasi
Revision: Jianer Chen
38 3-D Matching has no PTAS
In today's lecture, we study the approximability of the 3-D Matching
problem. Recall that a matching M in a set T of triples is a subset of T
such that no two triples in M have the same coordinate at any dimension.
3-D Matching
Input: a set S  X  Y  Z of triples
Output: a matching M in S with jM j maximized
According to Algorithm 30.1 Apprx3D-Second and Theorem 30.6, the
3-D Matching problem is in the class APX-PB. We show below that the
3-D Matching problem is ApxPB-complete.
We construct an E-reduction from an ApxPB-complete problem, the 3-
Occurrence Max-3Sat, to the 3-D Matching problem.
Let S be a set of clauses fC1; : : :; Cmg on boolean variables fx1; : : :; xn g
in which each clause contains at most three literals and each variable xi
appears, either as xi or as xi , at most 3 times in S . The set S is an instance
of the 3-Occurrence Max-3Sat problem. We construct an instance TS
of the 3-D Matching problem based on S .
We will use graphs to represent the triples in TS . Each triple will be
given as a triangle whose three vertices correspond to the three components
of the triple. Therefore, if two triples have a common component, then the
two corresponding triangles will have a shared vertex.
For each boolean variable u in fx1; : : :; xn g we have a ring structure. If
u has three occurrences in S , then the ring structure of u consists of six
triples, connected as in Figure 8(a). Similarly, if u has two occurrences or
one occurrence in S , then the ring structure of u consists of four triples or
two triples, connected as in Figure 8(b) and Figure 8(c), respectively.
In the following, we rst consider the case that the boolean variable u
has three occurrences in the set S . The discussion for the cases that u
205
 u[1,k] u[1,k]
u[1,k]
u[3,k] u[1,k]

u[2,k] u[1,k]

u[3,k] u[2,k]
u[1,k]
u[2,k] u[2,k]
(a) # occurrences of u = 3 (b) # occurrences of u = 2 (c) # occurrences of u = 1

Figure 8: The ring structure for the boolean variable u

has two occurrences or one occurrence in S is very similar, we will brie y

describe them after we complete the discussion on the case that u has three
occurrences.
For the boolean variable u that has three occurrences in the set S , there
are four identical rings of six triples. For k = 1; : : :; 4, the kth ring has its
outer vertices labeled by u[1; k], u[1; k], u[2; k], u[2; k], u[3; k], and u[3; k]
(see Figure 8). For each i = 1; 2; 3, the four vertices u[i; 1], u[i; 2], u[i; 3] and
u[i; 4] are connected by three new triples
(u[i; 1]; u[i; 2]; u1[i]); (u[i; 3]; u[i; 4]; u2[i]); (u1[i]; u2[i]; u[i])
in a binary tree manner. Similarly, for each i = 1; 2; 3, the four vertices
u[i; 1], u[i; 2], u[i; 3] and u[i; 4] are connected by three new triples
(u[i; 1]; u[i; 2]; u1[i]); (u[i; 3]; u[i; 4]; u2[i]); (u1[i]; u2[i]; u[i])
in a binary tree manner. Figure 9 shows the four rings and the three new
triples connecting the vertices u[1; 1], u[1; 2], u[1; 3] and u[1; 4]. Note that
the new triples connecting the other 20 triples in the rings are not shown in
Figure 9.
The triples contained in each ring will be called ring triples. The triples
(u[i; 1]; u[i; 2]; u1[i]); (u[i; 3]; u[i; 4]; u2[i]);
(u[i; 1]; u[i; 2]; u1[i]); (u[i; 3]; u[i; 4]; u2[i])
for i = 1; 2; 3, will be called leaf triples, and the triples
(u1[i]; u2[i]; u[i]) (u1[i]; u2[i]; u[i])

206
 u[1]
u1[1] u2[1]

u[1,1] u[1,2] u[1,3] u[1,4]

u[3,1] u[1,1] u[3,2] u[1,2] u[3,3] u[1,3] u[3,4] u[1,4]

u[3,1] u[2,1] u[3,2] u[2,2] u[3,3] u[2,3] u[3,4] u[2,4]

u[2,1] u[2,2] u[2,3] u[2,4]

Figure 9: The set Tu of triples for the boolean variable u

for i = 1; 2; 3, will be called root triples. Moreover, a triple will be called a

positive triple if it contains a component labeled as u[
] or u[
;
], and a triple
will be called a negative triple if it contains a component labeled as u[
] or
u[
;
]. Note that by this denition, every triple constructed above is either
a positive triple or a negative triple.
Therefore, there is a set Tu of 42 triples corresponding to each boolean
variable u in fx1 ; : : :; xn g: 24 of them are ring triples, 12 of them are leaf
triples, and 6 of them are root triples.
To make the set Tu a valid set of triples, i.e., a subset of X  Y  Z ,
we must label the vertices in Tu with X , Y , or Z properly. All trees will
be labeled identically, so we only describe the labeling for the tree rooted
at u[1]. Label u[1] with X , label u1[1] with Y and u2[1] with Z , and label
u[1; 1] with Z , u[1; 2] with X , u[1; 3] with X , and u[1; 4] with Y . Note that
for each xed ring, this labeling process labels all outer vertices of the ring
with the same symbol. Thus, the inner vertices in the ring can be properly
labeled using the other two symbols. It is not hard to verify that in this
labeling process, no triangle in Tu has two vertices labeled with the same
symbol. Therefore, the set Tu represents a set of triples.
We rst study the matching problem of the set Tu . Note that the set Tu
is also an instance of the 3-D Matching problem.
It is easy to check that the following two sets Mu+ and Mu in the set Tu
are matchings in Tu .
The set Mu+ consists of: (1) the 12 ring triples that contain u[i; k],
for i = 1; 2; 3 and k = 1; 2; 3; 4, respectively; (2) the 6 leaf triples
(u[i; 1]; u[i; 2]; u1[i]), and (u[i; 3]; u[i; 4]; u2[i]), for i = 1; 2; 3; and

207
 (3) the 3 root triples (u1[i]; u2[i]; u[i]), for i = 1; 2; 3.
The set Mu consists of: (1) the 12 ring triples that contain u[i; k],
for i = 1; 2; 3 and k = 1; 2; 3; 4, respectively; (2) the 6 leaf triples
(u[i; 1]; u[i; 2]; u1[i]), and (u[i; 3]; u[i; 4]; u2[i]), for i = 1; 2; 3; and
(3) the 3 root triples (u1[i]; u2[i]; u[i]), for i = 1; 2; 3.
Each of the matchings Mu+ and Mu contains 21 triples. The two match-
ings Mu+ and Mu will be called the canonical matchings in Tu .
Lemma 38.1 The canonical matchings Mu+ and Mu are maximum match-
ings in Tu .
proof. If we regard the set Tu as a graph, then a matching in Tu corre-
sponds to a set of disjoint triangles in the graph. We rst count the number
of vertices in this graph.
Each ring contains 12 dierent vertices, each leaf triple adds a new vertex,
and each root triple adds another new vertex. Since there are 4 rings, 12
leaf triples, and 6 root triples, we conclude that there are totally 66 vertices
in the graph.
Since 66 vertices can make at most 22 disjoint triangles, the number of
triples in a maximum matching in Tu is at most 22. Now suppose that Mu is
a matching of 22 triples in Tu . Then every vertex is contained in a triangle
in Mu . In particular, the six vertices labeled with u[i] and u[i], i = 1; 2; 3,
should appear in Mu . Since the six root triples are the only triples that
contain these vertices, all these six root triples should be in the matching
Mu . In consequence, no leaf triples can be in the matching Mu since every
leaf triple shares a vertex with a root triple. Now by the structure of the
rings, each ring can have at most 3 triples in Mu . Thus, the matching Mu
contains at most 12 ring triples. Summarizing all these, we derive that the
matching Mu would contain at most 18 triples, contradicting the assumption
that Mu contains 22 triples. Therefore, no matching in Tu can contain 22
triples.
Since the canonical matchings Mu+ and Mu each contains 21 triples in
Tu , we conclude that the canonical matchings are maximum matchings in
the set Tu .
Let Mu be a matching in Tu . We call a triple matched (by Mu ) if it is
contained in Mu . Otherwise, we say that the triple is unmatched.

208
Lemma 38.2 Let Mu be a matching in the set Tu. If Mu is not a canonical
matching, then Mu is not maximum.
proof. Suppose that Mu is a maximum matching in Tu . We show that
Mu must be one of the canonical matchings Mu+ and Mu .
By Lemma 38.1, jMu j = 21. Let ru , lu , and tu be the number of ring
triples, leaf triples, and root triples in Mu , respectively. Then
jMuj = 21 = ru + lu + tu
Since each matched root triple must be connected with two unmatched leaf
triples and since each leaf triple is connected with exactly one root triple,
we must have
tu  b(12 lu )=2c (17)
Since each matched leaf triple must be connected with two unmatched ring
triples and since each ring triple is connected with exactly one leaf triple,
we must have
ru  24 2lu (18)
Another trivial upper bound for ru is 12 since each ring can have at most 3
matched ring triples. This gives us
jMu j = ru + lu + tu
 minf24 lu + b(12 lu)=2c; 12 + lu + b(12 lu)=2cg
From this relation, it is easy to verify that in order to make jMu j = 21, we
must have lu = 6. Combining lu = 6 and jMu j = 21 together with Equations
(17) and (18), we also get tu = 3 and ru = 12.
From ru = 12, we derive that each ring in Tu must have exactly three
ring triples in Mu . Thus, each ring either has all its three positive triples
in Mu but none of its negative triples in Mu , or has all its three negative
triples in Mu but none of its positive triples in Mu .
We show that it is impossible that one ring has all its positive ring triples
in Mu while another ring has all its negative ring triples in Mu .
If the rst ring has all its positive ring triples in Mu while the second
ring has all its negative ring triples in Mu , then none of the six leaf triples
that are connected to the ring triples in the rst and the second rings can
be in the matching Mu . Since Mu contains six leaf triples and Tu has totally
12 leaf triples, all six leaf triples that are connected to ring triples in the
third and the fourth rings must be in Mu . But this is impossible because
209
it would imply that no ring triples in the third and the fourth rings are in
Mu . This proves that we must either have all positive ring triples in the
rst and the second rings in Mu or have all negative ring triples in the rst
and the second rings in Mu . Similarly, either all positive ring triples in the
third and the fourth rings are in Mu or all negative triples in the third and
the fourth rings are in Mu .
Now suppose that the rst and the second rings have all their positive
ring triples in Mu while the third and the fourth rings have all their negative
ring triples in Mu . Since the matching Mu has 6 leaf triples, the 3 negative
leaf triples connecting to triples in the rst and the second rings and the 3
positive leaf triples connecting to triples in the third and the fourth rings
must be in the matching Mu . However, it would imply that none of the root
triples can be in the matching Mu , contradicting the fact that the matching
Mu contains 3 root triples.
Therefore, we must either have all positive ring triples in Mu but no
negative ring triples in Mu , or have all negative ring triples in Mu but no
positive ring triples in Mu . Whenever this is decided, the six leaf triples and
the three root triples in Mu are uniquely determined. In fact, if all positive
ring triples are in Mu , then Mu must be the canonical matching Mu , while if
all negative ring triples are in Mu , then Mu must be the canonical matching
Mu+ .
This completes the discussion on the set Tu of triples, where u is a
boolean variable in fx1 ; : : :; xng that has 3 occurrences in the given set S of
clauses.
If u is a boolean variable that has 2 occurrences in S , then 4 rings of
4 triples, which has the structure shown in Figure 8(b), are used. Four
binary tree structures are constructed by adding 8 leaf triples and 4 root
triples. Thus, the set Tu of triples corresponding to u contains 28 triples.
There are two canonical matchings Mu+ and Mu of 14 triples in Tu such
that Mu+ contains all negative root triples but no positive root triples, while
Mu contains all positive root triples but no negative root triples. Moreover,
Mu+ and Mu are the only maximum matchings in the set Tu .
Similarly, if u is a boolean variable that has 1 occurrence in S , then 4
rings of 2 triples, which has the structure shown in Figure 8(c), are used.
Four binary tree structures are constructed by adding 4 leaf triples and 2
root triples. Thus, the set Tu of triples corresponding to u contains 14 triples.
There are two canonical matchings Mu+ and Mu of 7 triples in Tu such that
Mu+ contains all negative root triples but no positive root triples, while Mu
210
contains all positive root triples but no negative root triples. Moreover, Mu+
and Mu are the only maximum matchings in Tu .
We summarize these discussions into the following theorem.
Theorem 38.3 Let u be a boolean variable in fx1; : : :; xng such that u has
d occurrences in the set S , 1  d  3. Then one can construct a set Tu of
at most 42 triples with the following properties:
1. Tu has d positive root triples that contain the d components u[1], : : :,
u[d], respectively, and d negative root triples that contain the d com-
ponents u[1], : : :, u[d], respectively;
2. Tu has only two maximum matchings Mu+ and Mu , such that Mu
contains all d positive root triples but no negative root triples while
Mu+ contains all d negative root triples but no positive root triples.

211
 CPSC-669 Computational Optimization
Lecture #39, December 1, 1995
Lecturer: Professor Jianer Chen
Scribe: Balarama Varanasi
Revision: Jianer Chen
39 3-D Matching has no PTAS (contd.)
In the last lecture, we have shown the following theorem.
Theorem 39.1 Let u be a boolean variable in fx1; : : :; xng such that u has
d occurrences in the set S , 1  d  3. Then one can construct a set Tu of
at most 42 triples with the following properties:
1. Tu has d positive root triples that contain the d components u[1], : : :,
u[d], respectively, and d negative root triples that contain the d com-
ponents u[1], : : :, u[d], respectively;
2. Tu has only two maximum matchings Mu+ and Mu , such that Mu
contains all d positive root triples but no negative root triples while
Mu+ contains all d negative root triples but no positive root triples.
Now let us complete the construction of the set TS of triples, which is an
instance for the 3-D Matching problem, from the set S of clauses, which
is an instance for the 3-Occurrence Max-3Sat problem.
Let S be the set of clauses on the boolean variable set fx1 ; : : :; xng. The
set TS is the union of all sets Txi , i = 1; : : :; n, which satises the properties
stated in Theorem 39.1, plus the clause triples desribed as follows. For each
clause Ch = (u _ v _ w), where u, v , and w are literals in fx1; : : :; xn g and
we assume that this is the ith occurrence of u, the j th occurrence of v , and
the kth occurrence of w, the set TS contains three triples
(u[i]; y [h]; z [h]); (v [j ]; y [h]; z [h]); (w[k]; y [h]; z [h])
where y [h] and z [h] are two new symbols introduced for the clause Ch. Sim-
ilarly, if the clause Ch consists of 2 literals or 1 literal, the set TS introduces
two new symbols y [h] and z [h] and has 2 or 1 new triples. Figure 10 il-
lustrates this construction. This completes the construction of the set TS .

212
 y[h] z[h] y[h] z[h] y[h] z[h]

u[1] v[3] w[2] u[1] w[2] u[1]

the tree the tree the tree the tree the tree the tree
for u[1] for v[3] for w[2] for u[1] for w[2] for u[1]

Ch = (u v w) C h = (u w) C h = (u)

Figure 10: The clause triples in TS

Since each clause in S has at most 3 literals, the set TS contains at most
3m clause triples. Moreover, since each set Txi contains at most 42 triples,
we conclude that the set TS contains at most 42n + 3m triples. It is not
dicult to see that the set TS can be constructed from the set S of clauses
in polynomial time.
Let xi be a boolean variable. Each matching M in the set TS induces
a matching M=Txi in the set Txi of triples corresponding to the boolean
variable xi . We say that the matching M is a canonical matching for the set
TS if for all boolean variables xi, the induced matching M=Txi is a canonical
matching in the set Txi .
Lemma 39.2 Let M be a matching in the set TS . Then there is a canonical
matching M 0 in the set TS that contains at least as many triples as M does.
Moreover, the canonical matching M 0 can be constructed from the matching
M in polynomial time.
proof. For each variable xi , we consider the set Mi of clause triples in
M that contain an occurrence of xi .
If no clause triples in Mi contain a negative occurrence of xi , or Mi
is empty, then we replace the induced matching M=Txi by the canonical
matching Mx+i in Txi . Note that since the canonical matching Mx+i contains
only negative root triples but no positive root triples, this replacement still
gives a matching in TS . Moreover, since Mx+i is a maximum matching in Txi ,
this replacement does not decrease the number of triples in the matching.
Similarly, if no clause triples in Mi contain a positive occurrence of xi ,
then we replace the induced matching M=Txi by the canonical matching

213
Mxi in Txi , which gives a matching in TS that is at least as large as M .
Finally, suppose that the set Mi has a clause triple that contains a posi-
tive occurrence of xi and a clause triple that contains a negative occurrence
of xi . Then at least one positive root triple and at least one negative root
triple in the set Txi are not contained in the matching M . Consequently,
the induced matching M=Txi is not canonical, thus not maximum by The-
orem 39.1. Moreover, since the variable xi has at most three occurrences
in the set S , we have either at most one clause triple in Mi that contains a
positive occurrence of xi or at most one clause triple in Mi that contains a
negative occurrence of xi . Without loss of generality, we assume that only
one clause triple in Mi contains a positive occurrence of xi . Then we perform
the following operation: (1) delete the clause triple containing the positive
occurrence of xi , and (2) replace the induced matching M=Txi in Txi by
the canonical matching Mxi . Since after deleting the unique clause triple
containing the positive occurrence of xi , the matching M contains no clause
triples containing positive occurrences of xi while the canonical matching
Mxi in Txi contains only positive root triples in Txi , we conclude that this
operation still gives a matching in the set TS . Moreover, since the induced
matching M=Txi is not maximum in Txi , replacing M=Txi by the maximum
matching Mxi in Txi increases the number of matched triples in Txi by at
least one, which can be used to make up the clause triple deleted from Mi .
In consequence, this operation replaces the induced matching M=Txi in Txi
by a canonical matching in Txi and does not descrease the number of triples
in the matching.
If we apply the above process to each of the boolean variables xi , i =
1; : : :; n, we will eventually get a canonical matching M 0 in the set TS such
that the matching M 0 is at least as large as the matching M . It is also
easy to verify that the canonical matching M 0 can be constructed from the
matching M in polynomial time.
For each boolean variable xi , let ni be the numberPof triples contained
in a maximum matching in the set Txi , and let N0 = ni=1 ni .
Now we are ready to construct a solution for the instance S of the 3-
Occurrence Max-3Sat problem based on a solution for the instance TS
of the 3-D Matching problem.
Lemma 39.3 Given a matching M in the set TS , an assignment M to the
boolean variables fx1; : : :; xn g can be constructed in polynomial time such
that M satises at least jM j N0 clauses in the set S .

214
proof. Let M be a matching in the set TS . By Lemma 39.2, we can
construct in polynomial time a canonical matching M 0 in TS such that jM j 
jM 0j.
Let Mc0 be the subset of M 0 such that Mc0 consists of all clause triples in
0
M . Then we have
jMc0j = jM 0j N0
No boolean variable xi can have both its positive occurrence and its negative
occurrence contained in the clause triples in Mc0 | otherwise, the induced
matching M 0 =Txi would not be canonical in Txi . Therefore, we can construct
an assignment M to the boolean variables xi , : : :, xn as follows: if Mc0 has
a clause triple that contains a positive occurrence of xi then M assigns
xi = 1; if Mc0 has a clause triple that contains a negative occurrence of xi
then M assigns xi = 0. For variables that have no occurrences in Mc0 , M
assigns them arbitrarily. By the construction of the clause triples, for each
clause Ch , at most one corresponding clause triple is contained in the set Mc0 .
Moreover, if a clause Ch has a corresponding clause triple in the set Mc0 , then
the assignment M sets the clause Ch true. In conclusion, the assignment
M satises at least jMc0j clauses in S . From jMc0 j + N0 = jM 0j  jM j, we
derive jMc j  jM j N0 . The lemma follows.
Lemma 39.4 Let Optsat(S ) be the optimal value of the instance S for the 3-
Occurrence Max-3Sat problem and let Opt3dm (TS ) be the optimal value
of the instance TS for the 3-D Matching problem. Then
Optsat(S ) = Opt3dm (TS ) N0
proof. Lemma 39.3 shows Optsat (S )  Opt3dm (TS ) N0.
Now suppose that  is an assignment to fx1; : : :; xn g that satises the
largest number of clauses in the set S . Without loss of generality, let the
clauses satised by  be C1, : : :, Ck , where k = Optsat (S ). Suppose that
the assignment  sets the literal li true in the clause Ci , for i = 1; : : :; k.
If  sets more than one literal in Ci true, pick any of them as li . Then
we construct a matching M in the set TS as follows. For i = 1; : : :k, we
pick the clause triple (li ; y [i]; z [i]). If li is a positive occurrence of a boolean
variable xj , then we also pick all triples in the canonical matching Mx+j in
the set Txj , and if li is a negative occurrence of a boolean variable xj , then
we pick all triples in the canonical matching Mxj in the set Txj . Note that
for each boolean variable xj , the assignment  either sets all its positive
215
occurrences true or sets all its negative occurrences true. Therefore, the
above selection of triples cannot result in any con ict. Moreover, since no
positive root triple in Txj is contained in Mx+j and no negative root triple
in Txj is contained in Mxj , the above selection of triples makes a matching
in the set TS . Finally, for those boolean variables xj with no occurrence
in fl1; : : :; lk g, we pick all the triples in the canonical matching Mx+j . This
constructs a canonical matching M in the set TS , and M contains k clause
triples in TS . That is,
jMj = k + N0 = Optsat(S ) + N0
Since Opt3dm (TS )  jMj, we derive Optsat (S )  Opt3dm (TS ) N0 . The
lemma is proved.
Now we can describe how one can construct a solution M to the instance
S of the 3-Occurrence Max-3Sat problem from a solution M to the
instance TS of the 3-D Matching problem. Recall that S is a set of m
clauses. Consider the following algorithm.
Algorithm 39.1 3DM-to-3SAT
Input: a matching M
in the set TS

Output: an assignment M to 1 fx ; : : :; xng
1. construct an assignment  to fx1; : : :; xng that
satisfies at least jM j N0 clauses in S ;
2. if  satisfies less than m=2 clauses in S
then construct an assignment M that satisfies
at least m=2 clauses in S
else let M be ;
3. output M .

By Lemma 39.3, the assignment  in step 1 can be constructed in poly-

nomial time. Moreover, Algorithm 31.1 ApprxMaxSat and Lemma 31.2 show
that an assignment that satises at least m=2 clauses in S can be constructed
in polynomial time. In consequence, Algorithm 39.1 3DM-to-3SAT runs in
polynomial time.
To study the relative errors, let Apx(M ) be the number of clauses in
S that are satised by the assignment M . By the construction of the
assignment M , we have
Apx(M )  maxfjM j N0 ; m=2g (19)
216
Let Esat (S; M ) be the relative error of the solution M to the instance S
of the 3-Occurrence Max-3Sat problem, and let E3dm(TS ; M ) be the
relative error of the solution M to the instance TS of the 3-D Matching
problem. Note that the set TS has at most 3m + 42n  126m triples. Thus,
jM j  126m. Combining this fact with Equation (19) and Lemma 39.4, we
have
Esat(S; M ) = Opt sat (S ) 1 = Optsat (S ) Apx(M )
Apx(M ) Apx(M )
 Opt sat ( S ) ( j M j N 0 = Optsat (S ) + N0 jM j
)
m=2 m=2
= Opt3dmm= (TS ) jM j  Opt3dm (TS ) jM j
2 jM j=252
= 252E3dm(TS ; M )
This shows that the reduction we constructed from the 3-Occurrence
Max-3Sat problem to the 3-D Matching problem is an E-reduction. We
conclude with the following theorem.
Theorem 39.5 The 3-Occurrence Max-3Sat problem is E-reducible to
the 3-D Matching problem.
By Theorem 37.7, the 3-Occurrence Max-3Sat problem is ApxPB-
complete. We get
Theorem 39.6 The 3-D Matching problem is ApxPB-complete. There-
fore, the 3-D Matching problem has no polynomial time approximation
scheme unless P = NP.
Remark 39.1 The set TS of triples constructed from the set S of clauses
in our E-reduction is actually an instance of a more restricted version of
the 3-D Matching problem. Note that in the construction of the set TS ,
each symbol in X [ Y [ Z appears in at most 3 triples in TS . In fact, all
symbols in the set Txi , i = 1; : : :; n, appear in at most 2 triples in TS , only
the symbols y [h] and z [h] introduced for the clause Ch , h = 1; : : :; m, may
appear in 3 triples in TS . We can naturally dene a problem called the
3-Occurrence 3-D Matching by requiring that in an instance of the 3-
D Matching problem, each symbol appears in at most 3 triples. The set
TS is an instance of the 3-Occurrence 3-D Matching problem. Thus,

217
our E-reduction constructed in these two lectures actually reduces the 3-
Occurrence Max-3Sat problem to the 3-Occurrence 3-D Matching
problem. In consequence, the 3-Occurrence 3-D Matching problem is
also ApxPB-complete.
Remark 39.2 There is another optimization problem Triangle Packing
whose ApxPB-completeness can be easily obtained from an E-reduction from
the 3-D Matching problem. Let G be a graph. A triangle in G consists
of three mutually adjacent vertices in G. Two triangles in G are disjoint if
they do not share any common vertex. The Triangle Packing problem
is formulated as follows.
Tirangle Packing
Input: a graph G
Output: a set S of disjoint triangles in G with jS j maximized
It is not very hard to see that if an instance S of the 3-D Matching problem
is given as a graph, as we did in the last lecture, then a matching in S is a set
of disjoint triangles in the graph. This observation leads to an E-reduction
from the 3-D Matching problem to the Triangle Packing problem. We
leave the details to the interested students. On the other hand, the best
polynomial time approximation ratio for the Triangle Packing problem
is 2. Therefore, the Triangle Packing problem is ApxPB-complete.

218
 CPSC-669 Computational Optimization
Lecture #40, December 4, 1995
Lecturer: Professor Jianer Chen
Scribe: Balarama Varanasi
Revision: Jianer Chen
40 Max-Cut is ApxPB-complete
The last problem we will study in this course is the Max-Cut problem. Let
G = (V; E ) be a graph. A cut of the graph G is a partition D = (V1; V2) of
the vertex set V of G. That is, V1 [ V2 = V and V1 \ V2 = . We say that
an edge e of G is crossing in the cut D if one end of e is in V1 and the other
end of e is in V2. The Max-Cut problem is dened as follows.
Max-Cut
Input: a graph G = (V; E )
Output: a cut (V1; V2) of G that maximizes the number of
crossing edges
While the Max-Cut problem is NP-hard, it has a very simple approxi-
mation algorithm, as shown below.
Algorithm 40.1 Large-Cut
Input: a graph G = (V; E ), where V = fv1; : : :; vn g
Output: a cut ( V1; V2) of the graph G
1. let V1 =  and V2 = ;
2. for i = 1 to n do
if vi has more adjacent vertices in V1 than in V2
then V2 = V2 [ fvig
else V1 = V1 [ fvig;
3. output (V1; V2).

Theorem 40.1 The approximation algorithm Large-Cut for the Max-Cut

problem has approximation ratio 2.
proof. In the algorithm, each time the vertex vi is considered, the edges
connecting vi to the vertices v1 , : : :, vi 1 are counted. According to the
219
algorithm, at least half of these edges become crossing edges. Therefore,
at the end of the algorithm, at least half of the edges of the graph G are
crossing edges. Since no cut can have the number of crossing edges larger
than the number of edges in the graph G, the theorem follows.
Remark 40.1 This simple algorithm provided the best approximation ratio
for the Max-Cut problem for over 20 years. Very recently (1994), the
approximation ratio has been improved to 1:14. The complete version of the
paper is still not available yet. Interested students may ask the instructor
for a copy of the preliminary version of the paper.
To show that the Max-Cut problem is ApxPB-hard. we construct an
E-reduction from a ApxPB-complete problem, the 5-Occurrence Max-
2Sat problem, to the Max-Cut problem.
Let S = fC1; : : :; Cmg be an instance of the 5-Occurrence Max-2Sat
problem. That is, S is a set of clauses of at most two literals in the boolean
variable set fx1; : : :; xn g and each variable xi appears, either as xi or as
xi , at most ve times in the set S . We construct an instance GS for the
Max-Cut problem, where GS = (VS ; ES ) is a graph.
The vertex set VS of the graph GS is
VS = fx1; x1; : : :; xn; xn; z; zg
where z is a new symbol.
For each i = 1; : : :; n, there are 10 multiple edges connecting the vertices
xi and xi, and there are 2m multiple edges connecting the vertices z and z.
These edges connecting xi and xi or z and z will be called the pairing edges
of the graph GS . For each clause Ci = (u _ w) of two literals in S , we have a
triangle consisting of three edges [u; w], [u; z ], and [w; z ]; and for each clause
Ci = (u) of one literal in S , we have two multiple edges connecting u and z .
These edges will be called the clause edges corresponding to the clause Ci .
Note that if a literal u appears in two dierent clauses Ci and Cj , then there
are two multiple incident clause edges connecting u and z , corresponding to
the two clauses Ci and Cj , respectively.
This completes the construction of the instance GS of the Max-Cut
problem from the instance S of the 5-Occurrence Max-2Sat problem.
It is clear that the graph GS can be constructed from the set S of clauses
in polynomial time.
A cut D = (V1; V2) of GS is regular if for any u 2 fx1 ; : : :; xn ; z g, one of
u and u is in the set V1 and the other is in the set V2. For a cut D of the
graph GS , denote by jDj the number of crossing edges in the cut D.
220
Lemma 40.2 There is a polynomial time algorithm that, given a cut D of
the graph GS , constructs a regular cut D0 of GS such that jDj  jD0j.
proof. Suppose D = (V1; V2), where V1 [ V2 = VS and V1 \ V2 = .
If for any boolean variable xi , both vertices xi and xi are in V1, then we
remove xi from V1 and add it to V2. We show this does not decrease the
number of crossing edges in the cut. In fact, since S is an instance of the
5-Occurrence Max-2Sat problem and there are at most 2 clause edges
incident to xi corresponding to each clause containing xi , there are at most
10 clause edges incident on xi . Since both xi and xi are in V1 , these clause
edges are the only edges incident on xi that may be crossing edges in the
cut D. On the other hand, there are 10 pairing edges connecting xi and xi ,
which are not crossing edges in the cut D. Therefore, moving the vertex
xi from V1 to V2 will convert all these 10 pairing edges incident on xi from
non-crossing edges into crossing edges and may convert at most 10 clause
edges incident on xi from crossing edges into non-crossing edges. No other
crossing edges in the cut are changed. In consequence, this process does not
decrease the number of crossing edges in the cut.
If both vertices z and z are in V1, then we move the vertex z from the
set V1 to the set V2. Again, since z is incident to 2m clause edges and to
2m pairing edges, moving z from V1 to V2 does not decrease the number of
crossing edges in the cut.
The case when both xi and xi , or both z and z , are in the set V2 can be
dealt with in a completely similar way.
Thus, applying this process on each boolean variable xi and on z gives a
regular cut D0 without decreasing the number of crossing edges. Moreover,
it is easy to verify that the cut D0 can be constructed from the cut D in
polynomial time.
Lemma 40.3 There is a polynomial time algorithm that, given a cut D
of the graph GS , constructs an assignment 0 to fx1; : : :; xn g such that 0
satises at least (jDj 10n 2m)=2 clauses in the set S .
proof. We rst convert the cut D into a regular cut D0 of GS such that
jDj  jD0j. By Lemma 40.2, the cut D0 can be constructed in polynomial
time.
Let D0 = (V1; V2). Then for each u 2 fx1 ; : : :; xn; z g, exactly one of u
and u is in the set V1. Thus, all (10n + 2m) pairing edges in GS are crossing
edges in the cut D0. Without loss of generality, we assume that the vertex

221
z is in the set V1 while the vertex z is in the set V2.
We let 0 be the assignment to fx1; : : :; xng that sets all literals in the
set V1 fz g true. Note that since D0 = (V1; V2) is a regular cut, 0 is always
a valid assignment to fx1; : : :; xng.
Now consider a crossing edge e in D0 that is a clause edge corresponding
to a clause Ci in the set S . If Ci = (u _ w) consists of 2 literals, then since
the crossing edge e is one of the three clause edges [u; w], [u; z ] and [w; z ]
corresponding to the clause Ci, exactly two of these three clause edges are
crossing edges. In particular, by our assumption that the vertex z is in the
set V2, at least one of the vertices u and w is in V1. In consequence, the
assignment 0 sets this literal true and satises the clause Ci. If Ci = (u)
consists of 1 literal, since the crossing edge e is one of the multiple clause
edges connecting u and z , both multiple clause edges connecting u and z
are crossing edges. Moreover, the vertex u is in the set V1 since we assume
that the vertex z is in the set V2. Thus the assignment 0 satises the
clause Ci . We conclude that in either case, there are exactly two clause
edges corresponding to Ci that are crossing edges in the cut D0, and the
assignment 0 satises the clause Ci .
Since there are jD0j 10n 2m crossing edges in D0 that are clause edges,
and no three of them correspond to the same clause in S , we conclude that
the assignment 0 satises at least
(jD0j 10n 2m)=2  (jDj 10n 2m)=2
clauses in the set S .
Lemma 40.4 Let Optsat(S ) be the optimal value of S , regarded as an in-
stance of the 5-Occurrence Max-2Sat problem, and let Optcut (GS ) be
the optimal value of GS , regarded as an instance of the Max-Cut problem,
then
Optsat(S ) = (Optcut (GS ) 10n 2m)=2
proof. Let D be a maximum cut of the graph GS . By Lemma 40.3, there
is an assignment that satises at least
(jDj 10n 2m)=2 = (Optcut (GS ) 10n 2m)=2
clauses in the set S . Consequently,
Optsat(S )  (Optcut (GS ) 10n 2m)=2

222
 Conversely, let  be an assignment to fx1; : : :; xn g that satises the
largest number of clauses in the set S . Construct a cut D = (V1; V2) of the
graph GS such that a literal u is in V1 if and only if the assignment  sets
u true. Moreover, the vertex z is in the set V1 and the vertex z is in the set
V2. It is easy to verify that the cut D is a regular cut. Thus, all 10n + 2m
pairing edges in GS are crossing edges in the cut D .
Moreover, suppose without loss of generality that the assignment  sat-
ises the clauses Ci, i = 1; : : :; k, in the set S , where k = Optsat (S ). Thus,
the assignment  makes at least one literal ui true in the clause Ci , so the
literal ui is in the set V1. Since the vertex z is in the set V2 , exactly two of
the clause edges corresponding to the clause Ci are crossing edges. Thus,
there are at least 2k = 2Optsat (S ) clause edges that are crossing edges in
the cut D. This implies that the number of crossing edges in the cut D is
at least 2Optsat (S )+10n +2m, which should not be larger than Optcut (GS ).
Consequently,
Optsat(S )  (Optcut (GS ) 10n 2m)=2
This completes the proof of the lemma.
Now we are ready to show how a solution D to the instance GS of the
Max-Cut problem, where D is a cut of the graph GS , can be transformed
into a solution D to the instance S of the 5-Occurrence Max-2Sat
problem, where D is an assignment to the boolean variables fx1 ; : : :; xng.
Consider the following algorithm.
Algorithm 40.2 CUT-to-2SAT
Input: a cut D
of the graph GS

Output: an assignment D to fx1; : : :; xng
1. construct an assignment 0 to fx1 ; : : :; xn g such that 0
satisfies at least (jDj 10n 2m)=2 clauses in S ;
2. if 0 satisfies less than m=2 clauses in S
then construct an assignment D that satisfies
at least m=2 clauses in S
else let D be ;
3. output D .

By Lemma 40.3, the assignment 0 in step 1 can be constructed in poly-

nomial time. Moreover, Algorithm 31.1 ApprxMaxSat and Lemma 31.2 show
that an assignment that satises at least m=2 clauses in S can be constructed
223
in polynomial time. In consequence, Algorithm 40.2 CUT-to-2SAT runs in
polynomial time.
To study the relative errors, let Apx(D ) be the number of clauses in
S that are satised by the assignment D . By the construction of the
assignment D , we have
Apx(D )  maxf(jDj 10n 2m)=2; m=2g (20)
Let Esat (S; D ) be the relative error of the solution D to the instance
S of the 5-Occurrence Max-2Sat problem, and let Ecut(GS ; D) be the
relative error of the solution D to the instance GS of the Max-Cut problem.
Since each clause in S results in at most 3 clause edges in GS , the number
of edges in the graph GS is bounded by 10n + 2m + 3m, which is bounded
by 25m. Thus, jDj  25m. Combining this fact with Equation (20) and
Lemma 40.4, we have
Esat(S; D) = Opt sat (S ) 1 = Optsat (S ) Apx(D )
Apx(D ) Apx(D )
 Optsat(S ) (jm= Dj 10n 2m)=2
2
= 2Optsat (S ) (jmDj 10n 2m)
= (2Optsat(S ) + 10n + 2m) jDj
m
= Optcut(GS ) jDj
m
 Optcut(GS ) jDj
jDj=25
 Opt 
= 25 cut (GS ) 1
jDj
= 25Ecut(GS ; D)
This shows that the reduction we constructed from the 5-Occurrence
Max-2Sat problem to the Max-Cut problem is an E-reduction. We con-
clude with the following theorem.
Theorem 40.5 The 5-Occurrence Max-2Sat problem is E-reducible to
the Max-Cut problem.
By Theorem 37.13, the 5-Occurrence Max-2Sat problem is ApxPB-
complete. Combining this with Theorem 40.1 and Theorem 40.5, we get
224
Theorem 40.6 The Max-Cut problem is ApxPB-complete. Therefore, the
Max-Cut problem has no polynomial time approximation scheme unless P
= NP.
To close the course, we point out that for most of the optimization
problems studied in this course, we have precisely classied each of them
into a proper class: some of them are polynomial time solvable, some of
them are NP-hard but have fully polynomial time approximation schemes,
some of them have polynomial time approximation schemes but have no
fully polynomial time approximation schemes unless P = NP, and some of
them have polynomial time approximation algorithms with constant ratio
but have no polynomial time approximation scheme unless P = NP.
There are some optimization problems that even do not have a poly-
nomial time approximation algorithm with constant ratio. Examples are
the Independent Set problem, the Clique problem, and the Traveling
Salesman problem. For example, recent research has shown that there is
a constant  > 0 such that the Independent Set has no polynomial time
approximation algorithm with approximation ratio n unless P = NP.
One problem for which we did not study the non-approximability is the

-TSP problem. Theorem 22.5 shows that the problem has a polynomial
time approximation algorithm of approximation ratio 1:5. Using the E-
reduction, we can show that the
-TSP problem is ApxPB-complete. In
fact, even a weaker version, the Traveling Salesman 1-2 problem (see
Lecture 17) is ApxPB-complete.

225
References
[1] A. V. Aho, J. E. Hopcropt, and J. D. Ullman, The Design and
Analysis of Computer Algorithms, Addison-Wesley, Reading, Mass.,
1974.
[2] S. Arora, C. Lund, R. Motwani, M. Sudan, and M. Szegedy,
Proof verication and hardness of approximation problems, Proc. 33rd
Ann. IEEE Symp. on the Foundation of Computer Science, (1992), pp.
14-23.
[3] S. Arora and S. Safra, Probabilistic checking of proofs: a new
characterization of NP, Proc. 33rd Ann. IEEE Symp. on the Foundation
of Computer Science, (1992), pp. 2-13.
[4] G. Ausiello, P. Crescenzi, and M. Protasi, Approximate solu-
tion of NP optimization problems, Theoretical Computer Science 150,
(1995), pp. 1-55.
[5] B. S. Baker, Approximation algorithms for NP-complete problems
on planar graphs, Journal of ACM 41, (1994), pp. 153-180.
[6] P. Berman and G. Schnitger, On the complexity of approximating
the independent set problem, Information and Computation 96, (1992),
pp. 77-94.
[7] P. Berman and V. Ramaiyer, Improved approximations for the
Steiner tree problem, Proc. 3rd Ann. ACM-SIAM Symp. on Discrete
Algorithms, (1992), pp. 325-334.
[8] L. Cai and J. Chen, On the amount of nondeterminism and the
power of verifying, SIAM Journal on Computing, to appear.
[9] L. Cai and J. Chen, On xed-parameter tractability and approx-
imability of NP-hard optimization problems, Journal of Computer and
System Sciences, to appear.
[10] L. Cai, J. Chen, R. Downey, and M. Fellows, On the structure
of parameterized problems in NP, Information and Computation 123,
(1995), pp. 38-49.

226
[11] J. Chen and D. K. Friesen, The complexity of 3-dimensional
matching, Tech. Report, Dept. Computer Science, Texas A&M Uni-
versity, (1995).
[12] J. Chen, S. P. Kanchi, and A. Kanevsky, On the complexity of
graph embeddings, Lecture Notes in Computer Science 709, (1993), pp.
234-245.
[13] N. Christofides, Worst-case analysis of a new heuristic for the
traveling salesman problem, Tech. Report, GSIA, Carnegie-Mellon Uni-
versity, (1976).
[14] E. G. Coffman, M. R. Garey, and D. S. Johnson, Approxi-
mation algorithms for bin packing { an updated survey, in Algorithm
Design for Computer System Design, (ed. G. Ausiello, M. Lucertini,
and P. Serani), Springer-Verlag, 1984.
[15] P. Crescenzi and V. Kann, A compendium of NP optimization
problems, Manuscript, (1995).
[16] P. Crescenzi and A. Panconesi, Completeness in approximation
classes, Information and Computation 93, (1991), pp. 241-262.
[17] G. B. Dantzig, Linear Programming and Extensions, Princeton Uni-
versity Press, Princeton, NJ, 1963.
[18] E. A. Dinits, Algorithm for solution of a problem of maximum ow
in a network with power estimation, Soviet Math. Dokl. 11, (1970), pp.
1277-1280.
[19] J. Edmonds, Paths, trees and owers, Canad. J. Math. 17, (1965),
pp. 449-467.
[20] J. Edmonds and R. M. Karp, Theoretical improvements in al-
gorithmic eciency for network ow problems, Journal of ACM 19,
(1972), pp. 248-264.
[21] R. Fagin, Generalized rst-order spectra and polynomial-time recog-
nizable sets, SIAM-AMS Proc., (1974), pp. 43-73.
[22] W. Fernandez de la Vega and G. S. Lueker, Bin packing can be
solved within 1+  in linear time, Combinatorica 1, (1981), pp. 349-355.

227
[23] L. R. Ford and D. R. Fulkerson, Flows in Networks, Princeton
University Press, Princeton, NJ, 1962.
[24] D. K. Friesen, Tighter bounds for the multit processor scheduling
algorithm, SIAM Journal on Computing 13, (1984), pp. 170-181.
[25] M. R. Garey and D. S. Johnson, Strong NP-completeness results:
motivation, examples, and implications, Journal of ACM 25, (1978),
pp. 499-508.
[26] M. R. Garey and D. S. Johnson, Computers and Intractability: A
Guide to the Theory of NP-completeness, Freeman, San Fransico, CA,
1979.
[27] M. X. Goemans and D. P. Williamson, :878-approximation algo-
rithms for MAX CUT and MAX 2SAT, Proc. 26st Ann. ACM Symp.
on Theory of Computing, (1994), pp. 422-431.
[28] R. L. Graham, Bounds for certain multiprocessing anomalies, Bell
Systems Technical Journal 45, (1966), pp. 1563-1581.
[29] M. Grigni, E. Koutsoupias, and C. Papadimitriou, An approx-
imation scheme for planar graph TSP, Proc. 36st Ann. IEEE Symp. on
the Foundation of Computer Science, (1995), to appear.
[30] D. S. Hochbaum, Approximation algorithms for the set covering
and vertex cover problems, SIAM Journal on Computing 3, (1982), pp.
555-556.
[31] D. S. Hochbaum and D. B. Shmoys, Using dual approximation
algorithms for scheduling problems: theoretical and practical results,
Journal of ACM 34, (1987), pp. 144-162.
[32] D. S. Hochbaum and D. B. Shmoys, A polynomial approximation
scheme for scheduling on uniform processors: using the dual approxi-
mation approach, SIAM Journal on Computing 17, (1988), pp. 539-551.
[33] I. Holyer, The NP-completeness of edge coloring, SIAM Journal on
Computing 10, (1981), pp. 718-720.
[34] J. E. Hopcroft and R. M. Karp, A n5=2 algorithm for maximum
matching in bipartite graphs, SIAM Journal on Computing 2, (1973),
pp. 225-231.
228
[35] O. H. Ibarra and C. E. Kim, Fast approximation algorithms for
the knapsack and sum of subset problems, Journal of ACM 22, (1975),
pp. 463-468.
[36] D. S. Johnson, Approximation algorithms for combinatorial prob-
lems, Journal of Computer and System Sciences 9, (1974), pp. 256-278.
[37] D. S. Johnson, The NP-completeness column: an ongoing guide,
Journal of Algorithms 13, (1992), pp. 502-524.
[38] V. Kann, Maximum bounded 3-dimensional matching is MAX SNP-
complete, Information Processing Letters 37, (1991), pp. 27-35.
[39] N. Karmakar, A new polynomial-time algorithm for linear program-
ming, Combinatorica 4, (1984), pp. 373-395.
[40] N. Karmakar and R. M. Karp, An ecient approximation scheme
for the one-dimensional bin packing problem, Proc. 23rd Ann. IEEE
Symp. on Foundation of Computer Science, (1982), pp. 312-320.
[41] D. Karger, R. Motwani, and G. D. S. Ramkumar, On approxi-
mating the longest path in a graph, Lecture Notes in Computer Science
709, (1993), pp. 421-432.
[42] A. V. Karzanov, Determining the maximum ow in the network
with the method of pre ows, Soviet Math. Dokl. 15, (1974), pp. 434-437.
[43] S. Khanna, R. Motwani, M. Sudan, and U. Vazirani, On syn-
tactic versus computational views of approximability, Proc. 35th Ann.
IEEE Symp. on Foundation of Computer Science, (1994), pp. 819-836.
[44] D. E. Knuth, The Art of Computer Programming. Volume III: Sort-
ing and Searching, Addison-Wesley, Reading, Mass., 1973.
[45] P. G. Kolaitis and M. N. Thakur, Logical denability of NP
optimization problems, Information and Computation 115, (1994), pp.
321-353.
[46] P. G. Kolaitis and M. N. Thakur, Approximation properties of
NP minimization classes, Journal of Computer and System Sciences 50,
(1995), pp. 391-411.
[47] E. L. Lawler, Combinatorial Optimization: Networks and Matroids,
Holt, Rinehart&Winston, 1976.
229
[48] H. W. Lenstra, Integer programming with a xed number of vari-
ables, Mathematics of Operations Research 8, (1983), pp. 538-548.
[49] R. J. Lipton and R. E. Tarjan, A separator theorem for planar
graphs, SIAM J. Appl. Math. 36, (1979), pp. 177-189.
[50] R. J. Lipton and R. E. Tarjan, Applications of a planar separator
theorem, SIAM Journal on Computing 9, (1980), pp. 615-627.
[51] C. Lund and M. Yannakakis, On the hardness of approximating
minimization problems, Journal of ACM 41, (1994), pp. 960-981.
p
[52] S. Micali and V. V. Vazirani, An O( jV j
jE j) algorithm for
nding maximum matching in general graphs, Proc. 21st Ann. IEEE
Symp. on the Foundation of Computer Science, (1980), pp. 17-27.
[53] B. Monien, How to nd long paths eciently, Annals of Discrete
Mathematics 25, (1985), pp. 239-254.
[54] R. Motwani, Lecture Notes on Approximation algorithms, Dept. of
Computer Science, Stanford University, 1995.
[55] C. H. Papadimitriou, Combinatorial Complexity, Addison-Wesley,
Reading, MA, 1993.
[56] C. H. Papadimitriou and K. Steiglitz, Combinatorial Optimiza-
tion: Algorithms and Complexity, Englewood Clis, NJ: Prentice Hall,
1982.
[57] C. H. Papadimitriou and M. Yannakakis, Optimization, ap-
proximation, and complexity classes, Journal of Computer and System
Sciences 43, (1991), pp. 425-440.
[58] C. H. Papadimitriou and M. Yannakakis, The traveling sales-
man problem with distances one and two, Mathematics of Operations
Research 18, (1993), pp. 1-11.
[59] C. H. Papadimitriou and M. Yannakakis, On limited nondeter-
minism and the complexity of the V-C dimension, Journal of Computer
and System Sciences, (1995), to appear.
[60] F. P. Preparata and M. I. Shamos, Computational Geometry:
An Introduction, Springer-Verlag, New York, 1985.

230
[61] S. Sahni, Algorithms for scheduling independent tasks, Journal of
ACM 23, (1976), pp. 116-127.
[62] S. Sahni and T. Gonzalez, P-complete approximation problems,
Journal of ACM 23, (1976), pp. 555-565.
[63] D. B. Shmoys, Computing near-optimal solutions to combinato-
rial optimization problems, DIMACS Series in Discrete Mathematics,
(1995), to appear.
[64] V. G. Vizing, On an estimate of the chromatic class of a p-graph (in
Russian), Diskret. Analiz 3, (1964), pp. 23-30.
[65] M. Yannakakis, On the approximation of maximum satisability,
Journal of Algorithms 17, (1994), pp. 475-502.
[66] D. Zuckerman, On unapproximable versions of NP-complete prob-
lems, SIAM Journal on Computing, (1995), to appear.

231
Contents
1 Introduction 1
2 Max-Flow Problem 4
3 Max-Flow Problem (Contd.) 6
4 Max-Flow Problem (Contd.) 9
5 Max-Flow Problem (Contd.) 14
5.1 Edmonds-Karp's Algorithm : : : : : : : : : : : : : : : : : : : 14
5.2 Multiple source-sink ow problem : : : : : : : : : : : : : : : : 16
5.3 Graph Matching : : : : : : : : : : : : : : : : : : : : : : : : : 16
6 Karzanov's Algorithm 18
7 Maximum matching on bipartite graphs 24
7.1 Max-Flow Min-Cut Theorem : : : : : : : : : : : : : : : : : : 24
7.2 Hopcroft and Karp's analysis : : : : : : : : : : : : : : : : : : 26
8 Maximum matching for general graphs 30
9 Theorems on maximum matching problem 33
10 Algorithms for maximum matching problem 43
11 Linear programming problem 47
12 Integer Linear Programming Problem 51
13 NP-hard optimization problems 55
14 The Knapsack problem 59
15 Approximating Knapsack 63
16 Approximating Processor Scheduling 67
17 Which optimization problem has a FPTAS? 72
18 Strong NP-hardness 78
i
19 Absolute approximability 83
20 Planar Independent Set 94
21
-TSP: rst algorithm 101
22
-TSP: Christodes algorithm 107
23 Bin Packing problem 112
24 The (; )-Bin Packing problem 118
25 Approximating Bin Packing 123
26 Multi-processor scheduling 130
27 Approximating multi-processor scheduling 135
28 More on multi-processor scheduling 139
29 Approximability with a constant ratio 144
30 3-dimensional matching 149
31 Maximum satisability 157
32 Probabilistically Checkable Proofs 164
33 Max-3Sat has no PTAS 169
34 Independent Set has no PTAS 175
35 Independent Set is not in APX 182
36 Vertex Cover has no PTAS 187
37 The E-reducibility and the class APX-PB 195
38 3-D Matching has no PTAS 205
39 3-D Matching has no PTAS (contd.) 212

ii
40 Max-Cut is ApxPB-complete 219

iii
List of Figures
1 The structure of a blossom : : : : : : : : : : : : : : : : : : : 36
2 A fan structure : : : : : : : : : : : : : : : : : : : : : : : : : : 86
3 In case vh and w miss a common color c0 : : : : : : : : : : : 87
4 Extending a c0-cs alternating path Ps from vs not ending at w 88
5 Extending a c0 -cs alternating path Ph from vh not ending at w 88
6 The minimum spanning tree T : : : : : : : : : : : : : : : : : 104
7
-TSP Example. : : : : : : : : : : : : : : : : : : : : : : : : : 106
8 The ring structure for the boolean variable u : : : : : : : : : 206
9 The set Tu of triples for the boolean variable u : : : : : : : : 207
10 The clause triples in TS : : : : : : : : : : : : : : : : : : : : : 213

iv