8.

I NTRACTABILITY I

‣ poly-time reductions
‣ packing and covering problems
‣ constraint satisfaction problems
‣ sequencing problems
‣ partitioning problems
‣ graph coloring
‣ numerical problems
Lecture slides by Kevin Wayne
Copyright © 2005 Pearson-Addison Wesley
h t t p : / / w w w . c s . p r i n c e t o n . e d u / ~w a y n e / k l e i n b e r g - t a r d o s

Abdur Rashid Tushar

Lecturer, CSE, BUET

Last updated on 27.07.23 06:10

 8. I NTRACTABILITY I

‣ poly-time reductions
‣ packing and covering problems
‣ constraint satisfaction problems
‣ sequencing problems
‣ partitioning problems
‣ graph coloring
‣ numerical problems
SECTION 8.1
Algorithm design patterns and antipatterns

Algorithm design patterns.

・Greedy.
・Divide and conquer.
・Dynamic programming.
・Duality.
・Reductions.
・Local search.
・Randomization.

Algorithm design antipatterns.

・NP-completeness. O(nk) algorithm unlikely.
・PSPACE-completeness. O(nk) certification algorithm unlikely.
・Undecidability. No algorithm possible.

3
Classify problems according to computational requirements

Q. Which problems will we be able to solve in practice?

A working definition. Those with poly-time algorithms.

von Neumann Nash Gödel Cobham Edmonds Rabin

(1953) (1955) (1956) (1964) (1965) (1966)

Turing machine, word RAM, uniform circuits, …

Theory. Definition is broad and robust.

constants tend to be small, e.g., 3 n 2

Practice. Poly-time algorithms scale to huge problems.

4
Classify problems according to computational requirements

Q. Which problems will we be able to solve in practice?

A working definition. Those with poly-time algorithms.

yes probably no

shortest path longest path

min cut max cut

2-satisfiability 3-satisfiability

planar 4-colorability planar 3-colorability

bipartite vertex cover vertex cover

matching 3d-matching

primality testing factoring

linear programming integer linear programming

7
Classify problems

Desiderata. Classify problems according to those that can be solved in

polynomial time and those that cannot.

input size = c + log k

Provably requires exponential time.
・Given a constant-size program, does it halt in at most k steps?
・Given a board position in an n-by-n generalization of checkers,
can black guarantee a win?
using forced capture rule

Frustrating news. Huge number of fundamental problems have defied

classification for decades.

8
Poly-time reductions

Desiderata′. Suppose we could solve problem Y in polynomial time.

What else could we solve in polynomial time?

Reduction. Problem X polynomial-time (Cook) reduces to problem Y if

arbitrary instances of problem X can be solved using:
・Polynomial number of standard computational steps, plus
・Polynomial number of calls to oracle that solves problem Y.
computational model supplemented by special piece
of hardware that solves instances of Y in a single step

Algorithm
instance I solution S to I
for Y
(of X)

Algorithm for X

9
Poly-time reductions

Desiderata′. Suppose we could solve problem Y in polynomial time.

What else could we solve in polynomial time?

Reduction. Problem X polynomial-time (Cook) reduces to problem Y if

arbitrary instances of problem X can be solved using:
・Polynomial number of standard computational steps, plus
・Polynomial number of calls to oracle that solves problem Y.

Notation. X ≤ P Y.

Note. We pay for time to write down instances of Y sent to oracle 

instances of Y must be of polynomial size.

Novice mistake. Confusing X ≤ P Y with Y ≤ P X.

10
Intractability: quiz 1

Suppose that X ≤ P Y. Which of the following can we infer?

A. If X can be solved in polynomial time, then so can Y.

B. X can be solved in poly time iff Y can be solved in poly time.

C. If X cannot be solved in polynomial time, then neither can Y.

D. If Y cannot be solved in polynomial time, then neither can X.

11
Intractability: quiz 2

Which of the following poly-time reductions are known?

A. FIND-MAX-FLOW ≤ P FIND-MIN-CUT. O(mn2) time: Dinitz’ algorithm

O(m) time: given max flow f,

B. FIND-MIN-CUT ≤ P FIND-MAX-FLOW. let A = set of nodes reachable from s in Gf

C. Both A and B.

D. Neither A nor B.

12
Poly-time reductions

Design algorithms. If X ≤ P Y and Y can be solved in polynomial time,

then X can be solved in polynomial time.

Establish intractability. If X ≤ P Y and X cannot be solved in polynomial time,

then Y cannot be solved in polynomial time.

Establish equivalence. If both X ≤ P Y and Y ≤ P X, we use notation X ≡ P Y.

In this case, X can be solved in polynomial time iff Y can be.

Bottom line. Reductions classify problems according to relative difficulty.

13
 8. I NTRACTABILITY I

‣ poly-time reductions
‣ packing and covering problems
‣ constraint satisfaction problems
‣ sequencing problems
‣ partitioning problems
‣ graph coloring
‣ numerical problems
SECTION 8.1
Independent set

INDEPENDENT-SET. Given a graph G = (V, E) and an integer k, is there

a subset of k (or more) vertices such that no two are adjacent?

Ex. Is there an independent set of size ≥ 6 ?

Ex. Is there an independent set of size ≥ 7 ?

independent set of size 6

15
Vertex cover

VERTEX-COVER. Given a graph G = (V, E) and an integer k, is there a

subset of k (or fewer) vertices such that each edge is incident to
at least one vertex in the subset?

Ex. Is there a vertex cover of size ≤ 4 ?

Ex. Is there a vertex cover of size ≤ 3 ?

independent set of size 6

vertex cover of size 4

16
Intractability: quiz 3

Consider the following graph G. Which are true?

A. The white vertices are a vertex cover of size 7.

B. The black vertices are an independent set of size 3.

C. Both A and B.

D. Neither A nor B.

17
Vertex cover and independent set reduce to one another

Theorem. INDEPENDENT-SET ≡ P VERTEX-COVER.

Pf. We show S is an independent set of size k iff V − S is a vertex cover
of size n – k.

independent set of size 6

vertex cover of size 4

18
Vertex cover and independent set reduce to one another

Theorem. INDEPENDENT-SET ≡ P VERTEX-COVER.

Pf. We show S is an independent set of size k iff V − S is a vertex cover
of size n – k.


・Let S be any independent set of size k.
・V − S is of size n – k.
・Consider an arbitrary edge (u, v) ∈ E.
・S independent ⇒ either u ∉ S, or v ∉ S, or both.
⇒ either u ∈ V − S, or v ∈ V − S, or both.
・Thus, V − S covers (u, v). ▪

19
Vertex cover and independent set reduce to one another

Theorem. INDEPENDENT-SET ≡ P VERTEX-COVER.

Pf. We show S is an independent set of size k iff V − S is a vertex cover
of size n – k.


・Let V − S be any vertex cover of size n – k.
・S is of size k.
・Consider an arbitrary edge (u, v) ∈ E.
・V − S is a vertex cover ⇒ either u ∈ V − S, or v ∈ V − S, or both.
⇒ either u ∉ S, or v ∉ S, or both.
・Thus, S is an independent set. ▪

20
Set cover

SET-COVER. Given a set U of elements, a collection S of subsets of U, and an

integer k, are there ≤ k of these subsets whose union is equal to U ?

Sample application.
・m available pieces of software.
・Set U of n capabilities that we would like our system to have.
・The ith piece of software provides the set Si ⊆ U of capabilities.
・Goal: achieve all n capabilities using fewest pieces of software.

U = { 1, 2, 3, 4, 5, 6, 7 }
Sa = { 3, 7 } Sb = { 2, 4 }
Sc = { 3, 4, 5, 6 } Sd = { 5 }
Se = { 1 } Sf = { 1, 2, 6, 7 }
k=2

a set cover instance

21
Intractability: quiz 4

Given the universe U = { 1, 2, 3, 4, 5, 6, 7 } and the following sets, which is the

minimum size of a set cover?

A. 1 U = { 1, 2, 3, 4, 5, 6, 7 }
B. 2 Sa = { 1, 4, 6 } Sb = { 1, 6, 7 }

C. 3 Sc = { 1, 2, 3, 6 } Sd = { 1, 3, 5, 7 }
Se = { 2, 6, 7 } Sf = { 3, 4, 5 }
D. None of the above.

22
Vertex cover reduces to set cover

Theorem. VERTEX-COVER ≤ P SET-COVER.

Pf. Given a VERTEX-COVER instance G = (V, E) and k, we construct a
SET-COVER instance (U, S, k) that has a set cover of size k iff G has a
vertex cover of size k.

Construction.
・Universe U = E.
・Include one subset for each node v ∈ V : Sv = {e ∈ E : e incident to v }.
a b

e7 e2 e4 U = { 1, 2, 3, 4, 5, 6, 7 }
e3
Sa = { 3, 7 } Sb = { 2, 4 }
f e6 c Sc = { 3, 4, 5, 6 } Sd = { 5 }
e5 Se = { 1 } Sf = { 1, 2, 6, 7 }
e1
e d

vertex cover instance (k set cover instance

= 2) (k = 2)
23
Vertex cover reduces to set cover

Lemma. G = (V, E) contains a vertex cover of size k iff (U, S, k) contains

a set cover of size k.

Pf.  Let X ⊆ V be a vertex cover of size k in G. “yes” instances of VERTEX-COVER

・Then Y = { Sv : v ∈ X } is a set cover of size k. ▪ are solved correctly

a b

e7 e2 e4 U = { 1, 2, 3, 4, 5, 6, 7 }
e3
Sa = { 3, 7 } Sb = { 2, 4 }
f e6 c Sc = { 3, 4, 5, 6 } Sd = { 5 }
e5 Se = { 1 } Sf = { 1, 2, 6, 7 }
e1
e d

vertex cover instance (k set cover instance

= 2) (k = 2)
24
Vertex cover reduces to set cover

Lemma. G = (V, E) contains a vertex cover of size k iff (U, S, k) contains

a set cover of size k.

Pf.  Let Y ⊆ S be a set cover of size k in (U, S, k). “no” instances of VERTEX-COVER
・Then X = { v : Sv ∈ Y } is a vertex cover of size k in G. ▪ are solved correctly

a b

e7 e2 e4 U = { 1, 2, 3, 4, 5, 6, 7 }
e3
Sa = { 3, 7 } Sb = { 2, 4 }
f e6 c Sc = { 3, 4, 5, 6 } Sd = { 5 }
e5 Se = { 1 } Sf = { 1, 2, 6, 7 }
e1
e d

vertex cover instance (k set cover instance

= 2) (k = 2)
25
 8. I NTRACTABILITY I

‣ poly-time reductions
‣ packing and covering problems
‣ constraint satisfaction problems
‣ sequencing problems
‣ partitioning problems
‣ graph coloring
‣ numerical problems
SECTION 8.2
Satisfiability

Literal. A Boolean variable or its negation.

Clause. A disjunction of literals.

Conjunctive normal form (CNF). A propositional

formula Φ that is a conjunction of clauses.

SAT. Given a CNF formula Φ, does it have a satisfying truth assignment?

3-SAT. SAT where each clause contains exactly 3 literals
(and each literal corresponds to a different variable).

yes instance: x1 = true, x2 = true, x3 = false, x4 = false

Key application. Electronic design automation (EDA).

27
Satisfiability is hard

Scientific hypothesis. There does not exist a poly-time algorithm for 3-SAT.

P vs. NP. This hypothesis is equivalent to P ≠ NP conjecture.

https://www.facebook.com/pg/npcompleteteens

28
 3-satisfiability reduces to independent set

Theorem. 3-SAT ≤ P INDEPENDENT-SET.

Pf. Given an instance Φ of 3-SAT, we construct an instance (G, k) of
INDEPENDENT-SET that has an independent set of size k = ⎜Φ ⎜ iff Φ is
satisfiable.

Construction.
・G contains 3 nodes for each clause, one for each literal.
・Connect 3 literals in a clause in a triangle.
・Connect literal to each of its negations.

k=3

29
 3-satisfiability reduces to independent set

Lemma. Φ is satisfiable iff G contains an independent set of size k = ⎜Φ ⎜.

Pf.  Consider any satisfying assignment for Φ.

・Select one true literal from each clause/triangle. “yes” instances of 3-SAT

・This is an independent set of size k = Φ . ▪

⎜ ⎜
are solved correctly

k=3

30
 3-satisfiability reduces to independent set

Lemma. Φ is satisfiable iff G contains an independent set of size k = ⎜Φ ⎜.

Pf.  Let S be independent set of size k.

・S must contain exactly one node in each triangle. “no” instances of 3-SAT

・Set these literals to true (and remaining literals consistently). are solved correctly

・All clauses in Φ are satisfied. ▪

k=3

31
Review

Basic reduction strategies.

・Simple equivalence: INDEPENDENT-SET ≡ P VERTEX-COVER.
・Special case to general case: VERTEX-COVER ≤ P SET-COVER.
・Encoding with gadgets: 3-SAT ≤ P INDEPENDENT-SET.

Transitivity. If X ≤ P Y and Y ≤ P Z, then X ≤ P Z.

Pf idea. Compose the two algorithms.

Ex. 3-SAT ≤ P INDEPENDENT-SET ≤ P VERTEX-COVER ≤ P SET-COVER.

32
DECISION, SEARCH, AND OPTIMIZATION PROBLEMS

Decision problem. Does there exist a vertex cover of size ≤ k ?

Search problem. Find a vertex cover of size ≤ k.
Optimization problem. Find a vertex cover of minimum size.

Goal. Show that all three problems poly-time reduce to one another.

33
SEARCH PROBLEMS VS. DECISION PROBLEMS

VERTEX-COVER. Does there exist a vertex cover of size ≤ k ?

FIND-VERTEX-COVER. Find a vertex cover of size ≤ k.

Theorem. VERTEX-COVER ≡ P FIND-VERTEX-COVER.

Pf. ≤ P Decision problem is a special case of search problem. ▪

Pf. ≥ P
To find a vertex cover of size ≤ k :
・Determine if there exists a vertex cover of size ≤ k.
・Find a vertex v such that G − { v } has a vertex cover of size ≤ k − 1.
(any vertex in any vertex cover of size ≤ k will have this property)
・Include v in the vertex cover.
・Recursively find a vertex cover of size ≤ k − 1 in G − { v }. ▪

delete v and all incident edges 34

OPTIMIZATION PROBLEMS VS. SEARCH PROBLEMS

FIND-VERTEX-COVER. Find a vertex cover of size ≤ k.

FIND-MIN-VERTEX-COVER. Find a vertex cover of minimum size.

Theorem. FIND-VERTEX-COVER ≡ P FIND-MIN-VERTEX-COVER.

Pf. ≤ P Search problem is a special case of optimization problem. ▪

Pf. ≥ P To find vertex cover of minimum size:

・Binary search (or linear search) for size k * of min vertex cover.
・Solve search problem for given k . ▪*

35
 8. I NTRACTABILITY I

‣ poly-time reductions
‣ packing and covering problems
‣ constraint satisfaction problems
‣ sequencing problems
‣ partitioning problems
‣ graph coloring
‣ numerical problems
SECTION 8.5
Hamilton cycle

HAMILTON-CYCLE. Given an undirected graph G = (V, E), does there exist a

cycle Γ that visits every node exactly once?

yes
37
Hamilton cycle

HAMILTON-CYCLE. Given an undirected graph G = (V, E), does there exist a

cycle Γ that visits every node exactly once?

1 1′

2 2′

3 3′

4 4′

no
38
Directed Hamilton cycle reduces to Hamilton cycle

DIRECTED-HAMILTON-CYCLE. Given a directed graph G = (V, E), does there exist

a directed cycle Γ that visits every node exactly once?

Theorem. DIRECTED-HAMILTON-CYCLE ≤ P HAMILTON-CYCLE.

Pf. Given a directed graph G = (V, E), construct a graph G′ with 3n nodes.

aout
a din
d
b v
bout vin v vout
e
c
ein
cout
directed graph G undirected graph G′

39
Directed Hamilton cycle reduces to Hamilton cycle

Lemma. G has a directed Hamilton cycle iff G′ has a Hamilton cycle.

Pf. 
・Suppose G has a directed Hamilton cycle Γ.
・Then G′ has an undirected Hamilton cycle (same order). ▪

Pf. 
・Suppose G′ has an undirected Hamilton cycle Γ ′.
・Γ ′ must visit nodes in G′ using one of following two orders:
…, black, white, blue, black, white, blue, black, white, blue, …
…, black, blue, white, black, blue, white, black, blue, white, …
・Black nodes in Γ ′ comprise either a directed Hamilton cycle Γ in G,
or reverse of one. ▪

40
3-satisfiability reduces to directed Hamilton cycle

Theorem. 3-SAT ≤ P DIRECTED-HAMILTON-CYCLE.

Pf. Given an instance Φ of 3-SAT, we construct an instance G of

DIRECTED-HAMILTON-CYCLE that has a Hamilton cycle iff Φ is satisfiable.

Construction overview. Let n denote the number of variables in Φ.

We will construct a graph G that has 2n Hamilton cycles, with each cycle
corresponding to one of the 2n possible truth assignments.

41
 Poly-time reductions

constraint satisfaction

3-SAT

INDEPENDENT-SET DIR-HAM-CYCLE 3-COLOR SUBSET-SUM

VERTEX-COVER HAM-CYCLE KNAPSACK

SET-COVER

packing and covering sequencing partitioning numerical

49
 8. I NTRACTABILITY I

‣ poly-time reductions
‣ packing and covering problems
‣ constraint satisfaction problems
‣ sequencing problems
‣ partitioning problems
‣ graph coloring
‣ numerical problems
SECTION 8.6
3-dimensional matching

3D-MATCHING. Given n instructors, n courses, and n times, and a list of the

possible courses and times each instructor is willing to teach, is it possible
to make an assignment so that all courses are taught at different times?

instructor course time

Wayne COS 226 TTh 11–12:20

Wayne COS 423 MW 11–12:20

Wayne COS 423 TTh 11–12:20

Tardos COS 423 TTh 3–4:20

Tardos COS 523 TTh 3–4:20

Kleinberg COS 226 TTh 3–4:20

Kleinberg COS 226 MW 11–12:20

Kleinberg COS 423 MW 11–12:20

51
3-dimensional matching

3D-MATCHING. Given 3 disjoint sets X, Y, and Z, each of size n and a set

T ⊆ X × Y × Z of triples, does there exist a set of n triples in T such that
each element of X ∪ Y ∪ Z is in exactly one of these triples?

X = { x1, x2, x3 }, Y = { y1, y2, y3 }, Z = { z1 , z2 , z3 }

T1 = { x1, y1, z2 }, T2 = { x1, y2, z1 }, T3 = { x1, y2, z2 }

T4 = { x2, y2, z3 }, T5 = { x2, y3, z3 },
T7 = { x3, y1, z3 }, T8 = { x3, y1, z1 }, T9 = { x3, y2, z1 }

an instance of 3d-matching (with n = 3)

Remark. Generalization of bipartite matching.

52
3-dimensional matching

3D-MATCHING. Given 3 disjoint sets X, Y, and Z, each of size n and a set

T ⊆ X × Y × Z of triples, does there exist a set of n triples in T such that
each element of X ∪ Y ∪ Z is in exactly one of these triples?

Theorem. 3-SAT ≤ P 3D-MATCHING.

Pf. Given an instance Φ of 3-SAT, we construct an instance of 3D-MATCHING
that has a perfect matching iff Φ is satisfiable.

54
 8. I NTRACTABILITY I

‣ poly-time reductions
‣ packing and covering problems
‣ constraint satisfaction problems
‣ sequencing problems
‣ partitioning problems
‣ graph coloring
‣ numerical problems
SECTION 8.7
3-colorability

3-COLOR. Given an undirected graph G, can the nodes be colored black,

white, and blue so that no adjacent nodes have the same color?

yes instance

63
Intractability: quiz 6

How difficult to solve 2-COLOR?

A. O(m + n) using BFS or DFS.

B. O(mn) using maximum flow.

C. Ω(2n) using brute force.

D. Not even Tarjan knows.

64
Application: register allocation

Register allocation. Assign program variables to machine registers so that

no more than k registers are used and no two program variables that are
needed at the same time are assigned to the same register.

Interference graph. Nodes are program variables; edge between u and v

if there exists an operation where both u and v are “live” at the same time.

Observation. [Chaitin 1982] Can solve register allocation problem iff

interference graph is k-colorable.

Fact. 3-COLOR ≤ P K-REGISTER-ALLOCATION for any constant k ≥ 3.

65
3-satisfiability reduces to 3-colorability

Theorem. 3-SAT ≤ P 3-COLOR.

Pf. Given 3-SAT instance Φ, we construct an instance of 3-COLOR that

is 3-colorable iff Φ is satisfiable.

66
 Poly-time reductions

constraint satisfaction

3-SAT

INDEPENDENT-SET DIR-HAM-CYCLE 3-COLOR SUBSET-SUM

VERTEX-COVER HAM-CYCLE KNAPSACK

SET-COVER

packing and covering sequencing partitioning numerical

72
 8. I NTRACTABILITY I

‣ poly-time reductions
‣ packing and covering problems
‣ constraint satisfaction problems
‣ sequencing problems
‣ partitioning problems
‣ graph coloring
‣ numerical problems
SECTION 8.8
My hobby

NP-Complete by Randall Munro

http://xkcd.com/287
Creative Commons Attribution-NonCommercial 2.5
74
Subset sum

SUBSET-SUM. Given n natural numbers w1, …, wn and an integer W, is there a

subset that adds up to exactly W ?

Ex. { 215, 215, 275, 275, 355, 355, 420, 420, 580, 580, 655, 655 }, W = 1505.
Yes. 215 + 355 + 355 + 580 = 1505.

Remark. With arithmetic problems, input integers are encoded in binary.

Poly-time reduction must be polynomial in binary encoding.

75
Subset sum

Theorem. 3-SAT ≤ P SUBSET-SUM.

Pf. Given an instance Φ of 3-SAT, we construct an instance of SUBSET-SUM

that has a solution iff Φ is satisfiable.

77
SUBSET SUM REDUCES TO KNAPSACK

SUBSET-SUM. Given n natural numbers w1, …, wn and an integer W, is there a

subset that adds up to exactly W ?

KNAPSACK. Given a set of items X, weights ui ≥ 0, values vi ≥ 0, a weight limit

U, and a target value V, is there a subset S ⊆ X such that:

Recall. O(n U) dynamic programming algorithm for KNAPSACK.

Challenge. Prove SUBSET-SUM ≤ P KNAPSACK.

Pf. Given instance (w1, …, wn, W) of SUBSET-SUM, create KNAPSACK instance:

81
 Poly-time reductions

constraint satisfaction

3-SAT

INDEPENDENT-SET DIR-HAM-CYCLE 3-COLOR SUBSET-SUM

VERTEX-COVER HAM-CYCLE KNAPSACK

SET-COVER

packing and covering sequencing partitioning numerical

88
Karp’s 20 poly-time reductions from satisfiability

Dick Karp (1972)

1985 Turing Award

89