NP-Completeness: 數學家眼中的 CS

數學家的特性:
1.很聰明但大多話說不清楚
–太聰明:腦袋會轉彎,簡單事想的太複雜,也說的太
複雜
–數學家:其實是聽的人程度太差
2.目標遠大,心中想的是全世界
–喜歡一次解決一大堆(甚至全世界)問題,而不是
一個問題
想像自己是數學家才有辦法理解!
數學家的目標:
寫一個程式,一次解決全世界的所有問題
(目標很好,但心很壞,想讓 CS 的都失業)
34-1x

34-1a
The Hamiltonian cycle problem
Input: G = (V, E)
A nondeterministic algorithm
Step 1: Guess a cycle H
Step 2: Verify whether H is
a Hamiltonian cycle
(i) all edges exist?
a Hamiltonian cycle H (ii) visit each vertex exactly once?

(return 1 if H is correct;
otherwise return 0.)
 Non-deterministic sort (A)

Step 1: Guess B as the answer

Step 2: /* verification
for i = 1 to n-1 /* sorted ???
if not (B[i] <= B[i+ 1]) else return 0;

for i = 1 to n /* each A[i] is in B ???

if B[i] is in A, remove it from A
else return 0;

return 1;

34-1x

NP
P
NP-C

會計算就一定會驗算
Example: (x4-6x+8)1/2 - 3x2+6 = 0

Which of the following is easier?

P: x = ??? (計算in polynomial time)
NP: x = 3.412 ??? (驗算in polynomial time)

34-1y
What dose "solve" mean? 34-1b
an algorithm that runs in polynomial time (P)

What is the target?

un-computable

Set of all problems whose answers

problems can not be verified
in O(nk) time
computable

problems whose answers

can be verified
(NP)
in O(nk) time

NP-Completeness (數學家眼中的 CS) - Review

目標: 寫一個程式,一次解決全世界的所有問題,
為 CS 的人帶來光明(解救所有 programmers)
何謂解決(定義標準): polynomial (easy , can be solved)
– P (problems "can be solved")
何謂全世界(決定對手): NP (problems (answers) "can" be verified)
NP
P
N P -C
*會計算就一定會驗算

表示 A 是一個對手
− A ∈ NP CS: I gave an O(n4) algo for A
−A∈P表示 A 被解決了 => Math: I proved A ∈ P
−何謂解決全世界: prove NP = P

34-1z
Reduction: 34-2a
O(T)
Problem A Problem B
reduction
Example:
max bipartite O(V + E)
maximum flow
matching reduction

G = (V, E)
1
1 1
1 1
s 1 t
1 1
1 1 1
1

34-2b
Partition Problem: S1 and S2 such that
S = (a1, a2, …, an)
Sum(S1) = Sum(S2)

3-Partition Problem:
S1, S2, S3 such that
S = (a1, a2, …, an)
Sum(S1) = Sum(S2) = Sum(S3)

O(n)
Partition problem 3-partition problem
reduction
S = (a1, a2, …, an) S’ = (a1, a2, …, an, Sum(S)/2)

S = (3, 5, 7, 9) S’ = (3, 5, 7, 9, 24/2)

= (3, 5, 7, 9, 12)
Coloring: Given G, assign color to each node such that 34-2c
adjacent nodes have different colors.
O(n)
3-coloring 4-coloring
reduction

3-coloring: Determine whether a given G can be colored

by using {0, 1, 2}.
4-coloring: Determine whether a given G can be colored
by using {0, 1, 2, 3}.

Reduction: 34-2d
O(T)
Problem A Problem B
reduction
* usually:Which,
easyA
orhard (or, as ?hard as)
B, is harder
A can be solved by solving B
B is more difficult
* may: hard  easy

embed the difficulty

Example:
O(n lg n)
Find(A, x) Binary Search(S, x)
O(n) sort A O(lg n)
(i) If A B then A: O(na + nb) 34-2e
? O(na) O(nb)
imply a solution of A
(ii) If A B then B: ?
O(na) O(nb) ?
imply nothing for B
(iii) If A B then
b
Ω(na) O(n ) ?
may imply difficulty of B
case 1. Ω(n10) O(n5)
case 2. Ω(n10) O(n11)
case 1: b < a  B: Ω(na)
/* e.g. B: O(n9) then A: O(n5) + O(n9) = O(n9)
case 2: b ≥ a  B: ?
/* may: hard  easy, or easy  hard

Polynomial Reduction is transitive

T1 p
A B T1 + T2 p
T2 p A C
B C

in sense of hardness (hardness is transitive)

B ≥ hard A
C ≥ hard A
C ≥ hard B
(解掉 C 就解掉 A 和 B)

34-2x
 Problem A Problem B 34-2f
O(T1) O(T2)
imply a solution of A
* A: O(T1 + T2) 數學家心中的 reduction 都
* Assume that T1 is polynomial 是 polynomial
A ∈ P if T2 is polynomial
解掉 B 就解掉 A
A ∈ P if B ∈ P

O(nk) 解掉 B 就解掉全世界 (NP)

* If A B, then A ∈ P if B ∈ P
(i.e., A can be solved by solving B.)
O(nk)
* If all NP B, then all NP ∈ P if B ∈ P
(i.e., all NP can be solved by only solving B.)
假設真 (i.e., NP = P if B ∈ P.)
的存在

Example: 說服 全班 去參加一個旅遊
(解決)(全世界)
method 1. 一個一個的說服 (by CS people)
method 2. 透過 reduction 減少對手
小明: 小王去我就去
小明 小王 (解決小王就解決小明)
(對付小王不必管小明)
全班: 康樂去我就去
所有人 康樂 (對付康樂不必管全世界)
(擒賊先擒王)

34-2y
What is the goal? 34-2g

solve all problems in NP (in polynomial time) at a time

(i.e., prove NP = P) A ∈ NP 表示 A 是一個對手
A∈P 表示 A 被解決了
How can "all problems" be solved at a time?
Idea: reduction NP
(1) find a problem A in NP such that A

A
all problems in NP can be I B
reduced to A in polynomial time H
(2) solve A in polynomial time C
G D
F E
such a problem A is NP-C
(if exists)

A is NP-C: 34-2h
(1) A is in NP
(2) all NP problems can be reduced to A in polynomial time

Assume that there exists an NP-C A. I A

B
If A can be reduced to B in polynomial time H
(1) all NP problems can also be reduced to B C
(2) B is NP-C G D
F E

all NP p B ≅ an NP-C p B
 Example: 說服 全班去參加一個旅遊
(解決)(全世界)
method 1. 一個一個的說服 (by CS people)

透過 reduction 減少對手
method 2.
小明: 小王去我就去
小明 小王 (解決小王就解決小明)
(對付小王不必管小明)
全班: 康樂去我就去
所有人 康樂 (康樂 is NP-C)
康樂: 班代去我就去
康樂 班代
所有人 班代 (班代 is also NP-C )

34-2y

A is NP-C: 34-2h
(1) A is in NP
(2) all NP problems can be reduced to A in polynomial time

Assume that there exists an NP-C A. I A

B
If A can be reduced to B in polynomial time H
(1) all NP problems can also be reduced to B C
(2) B is NP-C G D
(3) B is as hard as A F E
All NP-C problems are of the same difficulty.
(They can be reduced to each other.)
I A
all NP p B ≅ an NP-C p B ≅ all NP-C p B H
C B
If an NP-C is solved
G D
all NP (including all NP-C) are solved,
F E
A is NP-C: 34-3a
(1) A is in NP
(2) all NP problems can be reduced to A in polynomial time X
A is NP-H: only (2) I A
B
Assume that there exists an NP-C A. H
C
If A can be reduced to B in polynomial time G D
(1) all NP problems can also be reduced to B F E
(2) B is NP-C
(3) B is as hard as A

If A can be reduced to an X ∉ NP in polynomial time

(1) all NP problems can be reduced to X
(2) X is NP-H, but not NP-C
(3) X is harder than A

Example: 說服 全班去參加一個旅遊
(解決)(全世界)
method 1. 一個一個的說服 (by CS people)

method 2. 透過 reduction 減少對手

全班: 康樂去我就去
所有人 康樂 (康樂 is NP-C)(& NP-H)
康樂: 班代去我就去
康樂 班代
所有人 班代 (班代 is NP-C )(& NP-H)
班代: 我最尊敬女朋友小美(外系),言聽計從
班代 小美
所有人 小美 (小美is NP-H, not NP-C )
34-2z
 NP-H: A,
???B, C, X, Y 34-3b
X
A, B, C
NP-C: ???
Y
All NP-C problems are of the same difficulty.
I A
But, all NP-H problems are not.
H B
C
If an NP-H or NP-C is solved
all NP (including all NP-C) are solved, G D
but not all NP-H F E

Goal: Solve all problems in NP at a time.

How: Solve a problem in NP-C?
QUESTION: Dose NP-C exist?

Note that it is impossible to solve a

problem in NP-H, but not in NP-C. Why?

(1) Find problems in NP-C

Two Efforts
- 1st: SAT /* all NP p SAT (Cook Thm., turing award)
SAT

3SAT
vertex cover Hamiltonian
xxxxx
cycle

xxxx xxxx xxxx

xxxx
xxxx xxxx
xxxx xxxx
xxxx
xxxx

(2) Solve an NP-C

- ????!!!!

34-3x
How to prove a problem A Example: 34-4a
is in NP-C (NP-H)? Prove 3-partition ∈ NP-C.
 Show that A is in NP.  Show that A is in NP.
give an O(nk)-time (i) guess S1, S2, S3
nondeterministic algo (ii) check S1 ∪ S2 ∪ S3 = S and
for A Sum(S1) = Sum(S2) = Sum(S3)
O(n lg n) time
O(nk) O(nk)
 Show that all in NP A.  Show that all in NP A.
(a) Find a problem Y ∈ NP-C (a) It is known 2-partition ∈ NP-C
O(nk) O(n)
(b) Show Y A (b) 2-partition 3-partition
O(nk) O(nk)
all NP Y A
O(nk) (omit  for NP-H)

NP-Completeness (數學家眼中的 CS) - Review

目標: 寫一個程式,一次解決全世界的所有問題,
為 CS 的人帶來光明 (解救所有 programmers)
何謂解決(定義標準): polynomial (easy , can be solved)
– P (problems "can be solved")
何謂全世界(決定對手): NP (problems "can" be verified)
–解決全世界: prove NP = P
如何下手:擒賊先擒王(直接解決敵軍中的大將軍)
– reduction:找出對手中的大魔王 NP-C
–解決大魔王
問題: 大魔王真的存在嗎? (yes, 1st - SAT, Turing award)
剩下的問題:大魔王可以被打敗嗎(can an N-PC be solved) ?
– NP = P or NP ≠ P? (unknown, but believe NP ≠ P )

結果:只帶來悲慘的消息(沒帶來光明,反而帶來黑暗)
–這世界到處都是不知如何對付的大魔王
34-4x
 NP
P
NP-C

34-4y

(a) NP-C: No one knows how to solve these problems. (Y/N) 34-4b
(No algorithms exist for these problems.)
(b) If we consider NP as an army, then
NP-C: ? NP-H but not NP-C: ?
If an NP-C surrenders, then
all NP too? all NP-C too? all NP-H too?
If an NP-H surrenders, then
all NP too? all NP-C too? all NP-H too?

(1) How to prove a problem is P? (5) How to prove NP = P?

(2) How to prove a problem is NP? (6) How to prove NP ≠ P?
(3) How to prove a problem is NP-C? (7) NP = P or NP ≠ P ?
(4) How to prove a problem is NP-H? (8) What do we learn ?
 Exercise
• 澄清關念,資格考題
– 找到的竟都是是非題 (decision problem)
– 全部照抄, 題目, 英文很差和我無關
– 也不為答案負責
– 老師在陪你作答, 不要只是在座位上看表演
(老師那麼老,年輕人要靠自己)

• Tips: whether a given problem is NP-H

– Guess: all SAT, but 2-SAT
– Guess: all coloring, but 2-coloring
– Guess: not on a DAG

34-6x

All simple paths: BFS 1 c 1 d

a 18 2 2
3 b 2 e
a

3 b c 1

5 d c 21 e 5 19 b 2 d

e c d 22 21 d e 21 b 4 e 4
7 6

24 e 23 e 6 e

34-4z
 1
All simple paths: DFS 1 c d
2
DFSDFS,
(usually, or BFS ???
save space) a 18 2
3 b 2 e
a

3 b c 1

5 d c 21 e 5 19 b 2 d

e c d 22 21 d e 21 b 4 e 4
7 6

24 e 23 e 6 e

34-4z’

Branch-and-bound
• Branch-and-bound: Brute-force + intelligent cuts
a

3 b cut by 5 c 1

5 d c 21 e 5 19 b 2 d

7
e c d d e b 4 e 4
7 6
cut by 7
e e 6 e

34-5x
 Pseudo-Polynomial:
If time is in the numeric value of an integer x,
we consider x as a lg2 x-bit integer (input size)
e.g. x = 60000, s = lg x = 16 bits
Example Example
input: N input: a, X
output: IsPrime(N) output: Xa
input siz : s = lg2 N input siz : s = lg2 a
Algorithm 1: Algorithm 1:
O(N) = O(2s) O(a) = O(2s)
exponential in s exponential in s
pseudo-polynomial pseudo-polynomial
Algorithm 2: Algorithm 2:
O(N1/2) = O(2s/2) O(lg a) = O(s)
pseudo-polynomial polynomial !!!
35-9y

Pseudo-Polynomial: 34-5a
polynomial in the numeric value of an integer
(exponential in the length (# of bits) of the integer)

If time is in the numeric value of an integer x,

we consider x as a lg2 x-bit integer (input size)
e.g. x = 60000, s = lg x = 16 bits
Examples: pseudo-polynomial polynomial
Counting sort - O(n + k)
GCD- O(lg b)
GCD- O(b)
Xa- O(lg a)
Xa- O(a)
Knapsack - O(nC)
(an NP-H problem)
 34-5b
Optimization Problems
* trivial
P * greedy
* DP

* brute-force (n ≤ 20 ~ 30)
exact solution
* B & B (n ≤ 30~100)
NP-hard * n > 100 ???
* approximation near-optimal
* heuristic solution

Remark: Pseudo-polynomial algorithms (DP, usually) are possible

for many NP-H problems (when inputs are small integers)