BLG 336E

Analysis of Algorithms II
Lecture 12:
Polynomial Time, Reductions
Independent Sets, Vertex Cover, Set Cover
Satisfiability, Hamiltonian Cycles

1
Recap
• Today we talked about s-t cuts and s-t flows.
• The Min-Cut Max-Flow Theorem says that minimizing
the cost of cuts is the same as maximizing the value of
flows.
• The Ford-Fulkerson algorithm does this!
• Find an augmenting path
• Increase the flow along that path
• Repeat until you can’t find any more paths and then you’re
done!
• An important algorithmic primitive!
• eg, assignment problems.
Polynomial-Time Reductions
 Classify Problems According to Computational Requirements

Q. Which problems will we be able to solve in practice?

A working definition. [von Neumann 1953, Godel 1956, Cobham 1964, Edmonds 1965, Rabin

1966]

Those with polynomial-time algorithms.

Yes Probably no
Shortest path Longest path
Matching 3D-matching
Min cut Max cut

2-SAT 3-SAT
Planar 4-color Planar 3-color
Bipartite vertex cover Vertex cover

Primality testing Factoring

4
 Classify Problems

Desiderata. Classify problems according to those that can be solved in

polynomial-time and those that cannot.

Probably requires exponential-time.

 Given a Turing machine, does it halt in at most k steps?
 Given a board position in an n-by-n generalization of chess,
can black guarantee a win?

Frustrating news. Huge number of fundamental problems have defied

classification for decades.

Today and Next week. Show that these fundamental problems are
"computationally equivalent" and appear to be different manifestations
of one really hard problem.

5
 Polynomial-Time Reduction

Reduction. Problem X polynomial-time reduces to problem Y if

arbitrary instances of problem X can be solved using:
 Polynomial number of standard computational steps, plus
 Polynomial number of calls to oracle that solves problem Y.

Notation. X  P Y.

Remarks.
 We pay for time to write down instances sent to black box 
instances of Y must be of polynomial size.
 Note: Cook reducibility (vs. Karp reducibility)

Means we can solve X in polynomial time IF we can

solve Y in polynomial time!

6
 Polynomial-Time Reduction

Purpose. Classify problems according to relative difficulty.

Design algorithms. If X  P Y and Y can be solved in polynomial-time,

then X can also be solved in polynomial time.

Establish intractability. If X  P Y and X cannot be solved in

polynomial-time, then Y cannot be solved in polynomial time.

Establish equivalence. If X  P Y and Y  P X, we use notation X  P Y.

7
Reduction By Simple Equivalence

Basic reduction strategies.

 Reduction by simple equivalence.
 Reduction from special case to general case.
 Reduction by encoding with gadgets.
 Independent Set

INDEPENDENT SET: Given a graph G = (V, E) and an integer k, is there a

subset of vertices S  V such that |S|  k, and for each edge at most
one of its endpoints is in S?

Ex. Is there an independent set of size  6? YES

Ex. Is there an independent set of size  7?
NO

independent set

9
 Vertex Cover

VERTEX COVER: Given a graph G = (V, E) and an integer k, is there a

subset of vertices S  V such that |S|  k, and for each edge, at least
one of its endpoints is in S?

Ex. Is there a vertex cover of size  4? YES

Ex. Is there a vertex cover of size  3? NO

vertex cover

10
 Vertex Cover and Independent Set

Claim. VERTEX-COVER P INDEPENDENT-SET.

Pf. We show S is an independent set iff V  S is a vertex cover.

independent set

vertex cover

11
Reduction from Special Case to General Case

Basic reduction strategies.

 Reduction by simple equivalence.
 Reduction from special case to general case.
 Reduction by encoding with gadgets.
 Set Cover

SET COVER: Given a set U of elements, a collection S1, S2, . . . , Sm of

subsets of U, and an integer k, does there exist a collection of  k of
these sets whose union is equal to U?

Sample application.
m available pieces of software.
Set U of n capabilities that we would like our system to have.
The ith piece of software provides the set Si  U of capabilities.
Goal: achieve all n capabilities using fewest pieces of software.

Ex:

U = { 1, 2, 3, 4, 5, 6, 7 }
k=2
S1 = {3, 7} S4 = {2, 4}
S2 = {3, 4, 5, 6} S5 = {5}
S3 = {1} S6 = {1, 2, 6, 7}

13
 Vertex Cover Reduces to Set Cover

Claim. VERTEX-COVER  P SET-COVER.

Pf. Given a VERTEX-COVER instance G = (V, E), k, we construct a set
cover instance whose size equals the size of the vertex cover instance.

Construction.
 Create SET-COVER instance:
– k = k, U = E, Sv = {e  E : e incident to v }
 Set-cover of size  k iff vertex cover of size  k. ▪

VERTEX COVER SET COVER

a b

e7 U = { 1, 2, 3, 4, 5, 6, 7 }
e2 e3 e4
k=2
Sa = {3, 7} Sb = {2, 4}
f e6 c
Sc = {3, 4, 5, 6} Sd = {5}
e1 e5 Se = {1} Sf= {1, 2, 6, 7}
k=2
e d

14
 Polynomial-Time Reduction

Basic strategies.
 Reduction by simple equivalence.
 Reduction from special case to general case.
 Reduction by encoding with gadgets.

15
 8.2 Reductions via "Gadgets"

Basic reduction strategies.

 Reduction by simple equivalence.
 Reduction from special case to general case.
 Reduction via "gadgets."
 Satisfiability

Literal: A Boolean variable or its negation. xi or xi

Clause: A disjunction of literals. C j  x1  x2  x3

Conjunctive normal form: A propositional   C1  C2  C3  C4

formula  that is the conjunction of clauses.

SAT: Given CNF formula , does it have a satisfying truth assignment?

3-SAT: SAT where each clause contains exactly 3 literals.

each corresponds to a different variable

Ex:  x1  x2  x 3    x1  x2  x 3    x2  x3    x1  x2  x 3 
Yes: x1 = true, x2 = true x3 = false.

17
 3 Satisfiability Reduces to Independent Set

Claim. 3-SAT  P INDEPENDENT-SET.

Pf. Given an instance  of 3-SAT, we construct an instance (G, k) of
INDEPENDENT-SET that has an independent set of size k iff  is
satisfiable.

Construction.
G contains 3 vertices for each clause, one for each literal.
Connect 3 literals in a clause in a triangle.
Connect literal to each of its negations.

x1 x2 x1

x2 x3 x1 x3 x2 x4

k=3    x1  x2  x3    x1  x2  x3    x1  x2  x4 
18
 3 Satisfiability Reduces to Independent Set

Claim. G contains independent set of size k = || iff  is satisfiable.

Pf.  Let S be independent set of size k.

S must contain exactly one vertex in each triangle.
 Set these literals to true. and any other variables in a consistent way

 Truth assignment is consistent and all clauses are satisfied.

Pf  Given satisfying assignment, select one true literal from each

triangle. This is an independent set of size k. ▪

x1 x2 x1

x2 x3 x1 x3 x2 x4

k=3    x1  x2  x3    x1  x2  x3    x1  x2  x4 
19
 Review

Basic reduction strategies.

 Simple equivalence: INDEPENDENT-SET  P VERTEX-COVER.
 Special case to general case: VERTEX-COVER  P SET-COVER.
 Encoding with gadgets: 3-SAT  P INDEPENDENT-SET.

Transitivity. If X  P Y and Y  P Z, then X  P Z.

Pf idea. Compose the two algorithms.

Ex: 3-SAT  P INDEPENDENT-SET  P VERTEX-COVER  P SET-COVER.

20
 Hamiltonian Cycle

HAM-CYCLE: given an undirected graph G = (V, E), does there exist a

simple cycle  that contains every node in V.

1 1'

2 2'

3 3'

4 4'

NO: bipartite graph with odd number of nodes.

21
 Directed Hamiltonian Cycle

DIR-HAM-CYCLE: given a digraph G = (V, E), does there exists a simple

directed cycle  that contains every node in V?

Claim. DIR-HAM-CYCLE  P HAM-CYCLE.

Pf. Given a directed graph G = (V, E), construct an undirected graph G'
with 3n nodes.

aout
a din
d
b v
bout vin v vout
e
c
ein
cout
G G'

22
 Directed Hamiltonian Cycle

Claim. G has a Hamiltonian cycle iff G' does.

Pf. 
 Suppose G has a directed Hamiltonian cycle .
 Then G' has an undirected Hamiltonian cycle (same order).

Pf. 
 Suppose G' has an undirected Hamiltonian cycle '.
 ' must visit nodes in G' using one of following two orders:
…, B, G, R, B, G, R, B, G, R, B, …
…, B, R, G, B, R, G, B, R, G, B, …
 Blue nodes in ' make up directed Hamiltonian cycle  in G, or
reverse of one. ▪

23
 3-SAT Reduces to Directed Hamiltonian Cycle

Claim. 3-SAT  P DIR-HAM-CYCLE.

Pf. Given an instance  of 3-SAT, we construct an instance of DIR-

HAM-CYCLE that has a Hamiltonian cycle iff  is satisfiable.

Construction. First, create graph that has 2n Hamiltonian cycles which

correspond in a natural way to 2n possible truth assignments.

24
 3-SAT Reduces to Directed Hamiltonian Cycle

Construction. Given 3-SAT instance  with n variables xi and k clauses.

 Construct G to have 2n Hamiltonian cycles.
 Intuition: traverse path i from left to right  set variable xi = 1.

x1

x2

x3

3k + 3 25
 3-SAT Reduces to Directed Hamiltonian Cycle

Construction. Given 3-SAT instance  with n variables xi and k clauses.

 For each clause: add a node and 6 edges.

C 1  x1 V x 2 V x 3 C 2  x1 V x2 V x3
clause node clause nod
e
s

x1

x2

x3

26
 3-SAT Reduces to Directed Hamiltonian Cycle

Claim.  is satisfiable iff G has a Hamiltonian cycle.

Pf. 
 Suppose 3-SAT instance has satisfying assignment x*.
 Then, define Hamiltonian cycle in G as follows:
– if x*i = 1, traverse row i from left to right
– if x*i = 0, traverse row i from right to left
– for each clause Cj , there will be at least one row i in which we are
going in "correct" direction to splice node Cj into tour

27
 3-SAT Reduces to Directed Hamiltonian Cycle

Claim.  is satisfiable iff G has a Hamiltonian cycle.

Pf. 
 Suppose G has a Hamiltonian cycle .
 If  enters clause node Cj , it must depart on mate edge.
– thus, nodes immediately before and after Cj are connected by an
edge e in G
– removing Cj from cycle, and replacing it with edge e yields
Hamiltonian cycle on G - { Cj }
 Continuing in this way, we are left with Hamiltonian cycle ' in
G - { C1 , C2 , . . . , Ck }.
 Set x*i = 1 iff ' traverses row i left to right.
 Since  visits each clause node Cj , at least one of the paths is
traversed in "correct" direction, and each clause is satisfied. ▪

28
 Longest Path

SHORTEST-PATH. Given a digraph G = (V, E), does there exists a simple

path of length at most k edges?

LONGEST-PATH. Given a digraph G = (V, E), does there exists a simple

path of length at least k edges?

Claim. 3-SAT  P LONGEST-PATH.

Pf 1. Redo proof for DIR-HAM-CYCLE, ignoring back-edge from t to s.

Pf 2. Show HAM-CYCLE  P LONGEST-PATH.

29
 Traveling Salesperson Problem

TSP. Given a set of n cities and a pairwise distance function d(u, v), is
there a tour of length  D?

All 13,509 cities in US with a population of at least 500

Reference: http://www.tsp.gatech.edu

30
 Traveling Salesperson Problem

TSP. Given a set of n cities and a pairwise distance function d(u, v), is
there a tour of length  D?

Optimal TSP tour

Reference: http://www.tsp.gatech.edu

31
 Traveling Salesperson Problem

TSP. Given a set of n cities and a pairwise distance function d(u, v), is
there a tour of length  D?

11,849 holes to drill in a programmed logic array

Reference: http://www.tsp.gatech.edu

32
 Traveling Salesperson Problem

TSP. Given a set of n cities and a pairwise distance function d(u, v), is
there a tour of length  D?

Optimal TSP tour

Reference: http://www.tsp.gatech.edu

33
 Traveling Salesperson Problem

TSP. Given a set of n cities and a pairwise distance function d(u, v), is
there a tour of length  D?

HAM-CYCLE: given a graph G = (V, E), does there exists a simple cycle
that contains every node in V?

Claim. HAM-CYCLE  P TSP.

Pf.
 Given instance G = (V, E) of HAM-CYCLE, create n cities with
distance function
 1 if (u, v)  E
d(u, v)  
 2 if (u, v)  E

 TSP instance has tour of length  n iff G is Hamiltonian. ▪

Remark. TSP instance in reduction satisfies -inequality.

34
NP and Computational Intractability
 P is a subset of NP
• Since it takes polynomial time to run the
program, just run the program and get a
solution
• But is NP a subset of P?
• No one knows if P = NP or not
• Solve for a million dollars!
– http://www.claymath.org/millennium-problems
– The Poincare conjecture is solved today
 Review

Basic reduction strategies.

 Simple equivalence: INDEPENDENT-SET  P VERTEX-COVER.
 Special case to general case: VERTEX-COVER  P SET-COVER.
 Encoding with gadgets: 3-SAT  P INDEPENDENT-SET.

Transitivity. If X  P Y and Y  P Z, then X  P Z.

Pf idea. Compose the two algorithms.

Ex: 3-SAT  P INDEPENDENT-SET  P VERTEX-COVER  P SET-COVER.

3
 Hamiltonian Cycle

HAM-CYCLE: given an undirected graph G = (V, E), does there exist a

simple cycle  that contains every node in V.

1 1'

2 2'

3 3'

4 4'

NO: bipartite graph with odd number of nodes.

3
 Directed Hamiltonian Cycle

DIR-HAM-CYCLE: given a digraph G = (V, E), does there exists a simple

directed cycle  that contains every node in V?

Claim. DIR-HAM-CYCLE  P HAM-CYCLE.

Pf. Given a directed graph G = (V, E), construct an undirected graph G'
with 3n nodes.

aout
a din
d
b v
bout vin v vout
e
c
ein
cout
G G'

3
 Directed Hamiltonian Cycle

Claim. G has a Hamiltonian cycle iff G' does.

Pf. 
 Suppose G has a directed Hamiltonian cycle .
 Then G' has an undirected Hamiltonian cycle (same order).

Pf. 
 Suppose G' has an undirected Hamiltonian cycle '.
 ' must visit nodes in G' using one of following two orders:
…, B, G, R, B, G, R, B, G, R, B, …
…, B, R, G, B, R, G, B, R, G, B, …
 Blue nodes in ' make up directed Hamiltonian cycle  in G, or
reverse of one. ▪

4
 3-SAT Reduces to Directed Hamiltonian Cycle

Claim. 3-SAT  P DIR-HAM-CYCLE.

Pf. Given an instance  of 3-SAT, we construct an instance of DIR-

HAM-CYCLE that has a Hamiltonian cycle iff  is satisfiable.

Construction. First, create graph that has 2n Hamiltonian cycles which

correspond in a natural way to 2n possible truth assignments.

41
 3-SAT Reduces to Directed Hamiltonian Cycle

Construction. Given 3-SAT instance  with n variables xi and k clauses.

 Construct G to have 2n Hamiltonian cycles.
 Intuition: traverse path i from left to right  set variable xi = 1.

x1

x2

x3

3k + 3 7
 3-SAT Reduces to Directed Hamiltonian Cycle

Construction. Given 3-SAT instance  with n variables xi and k clauses.

 For each clause: add a node and 6 edges.

C 1  x1 V x 2 V x 3 C 2  x1 V x2 V x3
clause node clause nod
e
s

x1

x2

x3

4
 Polynomial-Time Reductions

constraint satisfaction 3-SAT

Dick Karp
(1972)
1985 Turing
Award
INDEPENDENT SET DIR-HAM-CYCLE GRAPH 3-COLOR SUBSET-SUM

VERTEX COVER HAM-CYCLE PLANAR 3-COLOR SCHEDULING

SET COVER TSP

packing and covering sequencing partitioning numerical

4
Definition of NP
 The reducibility ‘tree’
• Richard Karp proved 21 problems to be NP
complete in a seminal 1971 paper
• Not that hard to read actually!
• Definitely not hard to read it to the point of
knowing what these problems are.
• karp's paper
 Amusing/tragic NP story
• Breaking up over NP
 Decision Problems

Decision problem.
X is a set of strings.
Instance: string s.
Algorithm A solves problem X: A(s) = yes iff s  X.

Polynomial time. Algorithm A runs in poly-time if for every string s,

A(s) terminates in at most p(|s|) "steps", where p() is some
polynomial.
length of s

PRIMES: X = { 2, 3, 5, 7, 11, 13, 17, 23, 29, 31, 37, …. }

Algorithm. [Agrawal-Kayal-Saxena, 2002] p(|s|) = |s|8.

48
 Definition of P

P. Decision problems for which there is a poly-time algorithm.

Problem Description Algorithm Yes No

Grade school
MULTIPLE Is x a multiple of y? 51, 17 51, 16
division

RELPRIME Are x and y relatively prime? Euclid (300 BCE) 34, 39 34, 51

PRIMES Is x prime? AKS (2002) 53 51

EDIT- Is the edit distance between Dynamic niether acgggt

DISTANCE x and y less than 5? programming neither ttttta

0 1 1  4 1 0 0 1
Is there a vector x that Gauss-Edmonds  2 4 2  ,  2   1 1 1 , 1
LSOLVE
satisfies Ax = b? elimination    
 0 3 15 36
 
0 1 1

1

49
 NP

Certification algorithm intuition.

 Certifier views things from "managerial" viewpoint.

 Certifier doesn't determine whether s  X on its own;
rather, it checks a proposed proof t that s  X.

Def. Algorithm C(s, t) is a certifier for problem X if for every string s, s  X iff there
exists a string t such that C(s, t) = yes.

"certificate" or "witness"

NP. Decision problems for which there exists a poly-time certifier.

C(s, t) is a poly-time algorithm and

|t|  p(|s|) for some polynomial p().

Remark. NP stands for nondeterministic polynomial-time.

50
 Certifiers and Certificates: Composite

COMPOSITES. Given an integer s, is s composite?

Certificate. A nontrivial factor t of s. Note that such a certificate

exists iff s is composite. Moreover |t|  |s|.

Certifier. boolean C(s, t) {

if (t  1 or t  s)
return false
else if (s is a multiple of t)
return true
else
return false
}

Instance. s = 437,669.
Certificate. t = 541 or 809. 437,669 = 541  809

Conclusion. COMPOSITES is in NP.

51
 Certifiers and Certificates: 3-Satisfiability

SAT. Given a CNF formula , is there a satisfying assignment?

Certificate. An assignment of truth values to the n boolean variables.

Certifier. Check that each clause in  has at least one true literal.

Ex.
 x1  x2  x3    x1  x2  x3    x1  x2  x4   x1  x3  x4 

instance s

x1 1, x2 1, x3  0, x4 1
certificate t

Conclusion. SAT is in NP.

52
 Certifiers and Certificates: Hamiltonian Cycle

HAM-CYCLE. Given an undirected graph G = (V, E), does there exist a

simple cycle C that visits every node?

Certificate. A permutation of the n nodes.

Certifier. Check that the permutation contains each node in V exactly

once, and that there is an edge between each pair of adjacent nodes
in the permutation.

Conclusion. HAM-CYCLE is in NP.

instance s certificate t

53
 P, NP, EXP

P. Decision problems for which there is a poly-time algorithm.

EXP. Decision problems for which there is an exponential-time algorithm.
NP. Decision problems for which there is a poly-time certifier.

Claim. P  NP.
Pf. Consider any problem X in P.
 By definition, there exists a poly-time algorithm A(s) that solves X.
 Certificate: t = , certifier C(s, t) = A(s). ▪

Claim. NP  EXP.
Pf. Consider any problem X in NP.
 By definition, there exists a poly-time certifier C(s, t) for X.
 To solve input s, run C(s, t) on all strings t with |t|  p(|s|).
 Return yes, if C(s, t) returns yes for any of these. ▪

54
 The Main Question: P Versus NP

Does P = NP? [Cook 1971, Edmonds, Levin, Yablonski, Gödel]

 Is the decision problem as easy as the certification problem?
 Clay $1 million prize.

EXP NP EXP
P P = NP

If P  NP If P = NP

would break RSA cryptography

(and potentially collapse economy)

If yes: Efficient algorithms for 3-COLOR, TSP, FACTOR, SAT, …

If no: No efficient algorithms possible for 3-COLOR, TSP, SAT, …

Consensus opinion on P = NP? Probably no.

55
NP-Completeness
 NP-Complete

NP-complete. A problem Y in NP with the property that for every

problem X in NP, X  p Y.

Theorem. Suppose Y is an NP-complete problem. Then Y is solvable in

poly-time iff P = NP.
Pf.  If P = NP then Y can be solved in poly-time since Y is in NP.
Pf.  Suppose Y can be solved in poly-time.
 Let X be any problem in NP. Since X  p Y, we can solve X in
poly-time. This implies NP  P.
 We already know P  NP. Thus P = NP. ▪

Fundamental question. Do there exist "natural" NP-complete problems?

57
 NP-Completeness

Observation. All problems below are NP-complete and polynomial

reduce to one another!

by definition of NP-completeness
CIRCUIT-SAT

3-SAT

INDEPENDENT SET DIR-HAM-CYCLE GRAPH 3-COLOR SUBSET-SUM

VERTEX COVER HAM-CYCLE PLANAR 3-COLOR SCHEDULING

SET COVER TSP

58
 NEXT LECTURE
 NP-Completeness
 Review

59