NP-COMPLETENESS

Nguyen An Khuong, CSE-HCMUT

HCMUT Advanced Algorithms Course (CO5127), Lecture on March 20, 2024

(Materials drawn from CLRS’22, Chapter 34)

1/ 34
Outline

Overview

Turing Machines, P, and NP

Reductions and NP-completeness

Some popular NP-complete problems

Further Reading and Reeferences

2/ 34
1. Overview

3/ 34
Overview 1: Hardness of computational problems

▶ Some are “tractable”: maximum element, shortest paths, LCS, etc.

▶ Some are provably intractable: Is a formula involving
+, =, <, ∃, ̸=, ∨, ∧, x1 , ..., xn satisfiable when interpreted over the integers?
Are two extended regular expressions equivalent?
▶ Some are “undecidable”: Are two programs equivalent, assuming the
programming language has some basic constructs, e.g., loops, integer types,
etc.? Is a formula involving +, =, <, ∃, ̸=, ∨, ∧, x1 , ..., xn satisfiable when
interpreted over the integers?
▶ Some are (very) likely intractable: longest simple path, 3-colorability, circuit
satisfiability, etc.

4/ 34
Overview 2: Polynomial vs. NP-complete
▶ Polynomial-time algorithm: There exists a constant c ∈ N such that the
algorithm has (worst-case) running time O (nc ), where n is the size of the
input.
▶ Examples:
▶ Polynomial: n; n2 log2 n; n3 ; √
n20
▶ Super-polynomial: nlog2 n ; 2 n ; 1.001n ; 2n ; n!
▶ Central Question: Which computational problems have polynomial-time
algorithms?
▶ Million-dollar question: Intriguing class of problems: NP-complete
problems.
▶ NP-complete problems: The class NP consists of problems where a
purported solution can be verified in polynomial time. The class NPC
consists of problems that are in NP and are as “hard” as any other problem
in NPC time
▶ It is unknown whether NPC problems have polynomial-time algorithms. 5/ 34
Overview 3: Polynomial vs. NP-complete
▶ Problem is in NPC means that it’s very unlikely that it can be solved
efficiently
▶ To be a good algorithm designer, you need to understand some of the basic
elements of the theory of NP-completeness.
▶ A polynomial-time algorithm for only one NP-complete problem would imply
polynomial-time algorithms for all problems in NP. (see Woeginger’s P vs
NP page: https://www.win.tue.nl/∼wscor/woeginger/P-versus-NP.htm)
▶ If a problem is NPC, focus your attention on special cases approximation
techniques (see CLRS’22 Chapter 35)
▶ Focus will be on decision problems (the answer is YES/NO) rather than
optimization problems:
▶ Decision problems are closely related to optimization problems, e.g., given a
graph G, vertices u, v , and integer k, does there exist a path of length ≤ k
from u to v ?
▶ Optimization version is at least as hard as decision version =⇒ decision
problem is hard implies optimization problem is hard 6/ 34
Overview 4: Some well-known Poly. and NPC Problems
▶ Polynomial: Informally, the class P consists of problems that are solvable in
polynomial time, specifically in time O(nk ) where n is the instance size, and
k is a constant.
▶ Shortest Path: Given a graph G, two vertices a and b of G, and an
integer k, does G have a simple a − b-path of length at most k ?
▶ Euler Tour: Given a graph G, does G have a cycle that traverses each
edge of G exactly once?
▶ 2-CNF SAT: Given a propositional formula F in 2-CNF, is F satisfiable?
▶ NP-complete:
▶ Longest Path: Given a graph G and an integer k, does G have a simple
path of length at least k?
▶ Hamiltonian Cycle: Given a graph G, does G have a simple cycle that
visits each vertex of G ?
▶ 3-CNF SAT: Given a propositional formula F in 3-CNF, is F satisfiable?
Example: (x ∨ ¬y ∨ z) ∧ (¬x ∨ z)∧ (¬y ∨ ¬z).
7/ 34
2. Turing Machines, P, and NP

8/ 34
Turing Machines, P, and NP: Decision probs and Encodings
▶ We are going to
▶ Formally define P, NP, and NP-complete (NPC), and then
▶ Learn how to show that a problem is NP-complete
▶ <Name of Decision Problem >
Input: <What constitutes an instance >
Question: <Yes/No question >
▶ We want to know which decision problems can be solved in polynomial time
- polynomial in the size of the input n.
▶ Assume a ”reasonable” encoding of the input
▶ Many encodings are polynomial-time equivalent; i.e., one encoding can be
computed from another in polynomial time.
▶ Important exception: unary versus binary encoding of integers.
▶ An integer x takes ⌈log2 x ⌉ bits in binary and x = 2log2 x bits in unary.

9/ 34
Turing Machines, P, and NP: Formal-language framework

We can view decision problems as languages.

▶ Alphabet Σ: a finite set of symbols. W.l.o.g., Σ = {0, 1}
▶ Language L over Σ : set of strings made with symbols from Σ : L ⊆ Σ∗
▶ Fix an encoding of instances of a decision problem Π into Σ
▶ Define the language LΠ ⊆ Σ∗ such that

x ∈ LΠ ⇔ x is a Yes-instance for Π.

10/ 34
Non-deterministic Turing
∗
Machine (NTM)
▶ input word x ∈ Σ placed on an ▶ Q: a finite, non-empty set of
infinite tape (memory) states
▶ read-write head initially placed on ▶ Γ: a finite, non-empty set of tape
the first symbol of x symbols
▶ computation step: if the machine ▶ ∈ Γ: blank symbol (the only
is in state s and reads a, it can symbol allowed to occur on the
move into state s ′ , writing b, and tape infinitely often)
moving the head into direction ▶ Σ ⊆ Γ\{b} : set of input symbols
D ∈ {L, R} if
▶ q0 ∈ Q : start state
((s, a), (s ′ , b, D)) ∈ δ.
▶ A ⊆ Q: set of accepting (final)
states
▶ δ ⊆ (Q\A×Γ)×(Q ×Γ×{L, R}) :
transition relation, where L (R)
stands for a move to the left
11/ 34
Accepted Language, Accept and Decide in poly. time
▶ Definition. An NTM accepts a word x ∈ Σ∗ if there exists a sequence of
computation steps starting in the start state and ending in an accepted state.
▶ Definition. The language accepted by an NTM is the set of words it
accepts.
▶ Video (The LEGO Turing Machine): https://youtu.be/cYw2ewoO6c4
▶ Definition. A language L is accepted in polynomial time by an NTM M if
▶ L is accepted by M, and
▶ there
 is a constant k such that for any word x ∈ L, the NTM M accepts x in
O |x |k computation steps.
▶ Definition. A language L is decided in polynomial time by an NTM M if
▶ there
 is a constant k such that for any word x ∈ L, the NTM M accepts x in
O |x |k computation steps, and
▶ there is a constant k ′ such that for any word x ∈ Σ∗ \L, on input x the NTM
 ′

M halts in a non-accepting state (Q\A) in O |x |k computation steps.
12/ 34
DTM and DTM equivalents

▶ Definition 5. A Deterministic Turing Machine (DTM) is a

Non-deterministic Turing Machine where the transition relation contains at
most one tuple ((s, a), (·, ·, ·)) for each s ∈ Q\A and a ∈ Γ.
▶ The transition relation δ can be viewed as a function
δ : Q\A × Γ → Q × Γ × {L, R}.
▶ ⇒ For a given input word x ∈ Σ∗ , there is exactly one sequence of
computation steps starting in the start state.
▶ Many computational models are polynomial-time equivalent to DTMs:
▶ Random Access Machine (RAM, used for algorithms in the textbook)
▶ variants of Turing machines (multiple tapes, infinite only in one direction, ...)
▶ ...

13/ 34
Definitions of P and NP
▶ Definition (P-class). P = {L ⊆ Σ∗ : there is a DTM accepting L in
polynomial time}
▶ Definition (NP-class). NP = {L ⊆ Σ∗ : there is an NTM accepting L in
polynomial time}
▶ Definition (coNP-class). coNP = {L ⊆ Σ∗ : Σ∗ \L ∈ NP}
coP?
▶ Theorem. P = {L ⊆ Σ∗ : there is a DTM deciding L in polynomial time}
▶ Proof (sketch). Need to show: if L is accepted by a DTM M in
polynomial time, then there is a DTM that decides L in polynomial time.
Idea: design a DTM M ′ that simulates M for c · nk steps, where c · nk is
the running time of M. (Note that this proof is nonconstructive: we might
not know the running time of M.)

14/ 34
NP and certificates: Non-deterministic choices

▶ An NTM for an NP-language L makes a polynomial number of

non-deterministic choices on input x ∈ L. We can encode these
non-deterministic choices into a certificate c, which is a polynomial-length
word. Now, there exists a DTM, which, given x and c, verifies that x ∈ L in
polynomial time.
▶ Thus, L ∈ NP iff there is a DTM V and for each x ∈ L there exists a
polynomial-length certificate c such that V (x , c) = 1, but V (y , ·) = 0 for
each y ∈
/ L.

15/ 34
CNF-SAT is in NP
▶ A CNF formula is a propositional formula in conjunctive normal form:
conjunction (AND) of clauses; each clause is a disjunction (OR) of literals;
each literal is a negated or unnegated Boolean variable.
▶ An assignment α : var(F ) → {0, 1} satisfies a clause C if it sets a literal of
C to true, and it satisfies F if it satisfies all clauses in F .
▶ CNF-SAT Problem:
Input: CNF formula F
Question: Does F have a satisfying assignment?
▶ Example: (x ∨ ¬y ∨ z) ∧ (¬x ∨ z) ∧ (¬y ∨ ¬z).
▶ Lemma. CNF-SAT ∈ NP.
▶ Proof. Certificate: assignment α to the variables. Given a certificate, it can
be checked whether all clauses are satisfied.
16/ 34
Brute-force algorithms for problems in NP
▶ Theorem. Every problem in NP can be solved in exponential time.
▶ Proof. Let Π be an arbitrary problem in NP. [Use certificate-based
definition of NP] We know that ∃ a polynomial p and a polynomial-time
verification algorithm V such that:
▶ for every x ∈ Π (i.e., every Yes-instance for Π)∃ string
c ∈ {0, 1}∗ , |c| ≤ p(|x |), such that V (x , y ) = 1, and
▶ for every x ∈/ Π (i.e., every No-instance for Π ) and every string
c ∈ {0, 1}∗ , V (x , c) = 0.
▶ Now, we can prove that there exists an exponential-time algorithm for Π
with input x :
▶ For each string c ∈ {0, 1}∗ with |c| ≤ p(|x |), evaluate V (x , c) and return
Yes if V (x , c) = 1.
▶ Return No.
▶ Running time: 2p(|x |) · nO(1) ⊆ 2O(2·p(|x |)) = 2O(p(|x |)) , but
nonconstructive. 17/ 34
3. Reductions and NP-completeness

18/ 34
Polynomial-time reduction and NP-completeness
▶ Definition. A language L1 is polynomial-time reducible to a language L2 ,
written L1 ≤P L2 , if there exists a polynomial-time computable function
f : Σ∗ → Σ∗ such that for all x ∈ Σ∗ , x ∈ L1 ⇔ f (x ) ∈ L2 .
▶ A polynomial time algorithm computing f is a reduction algorithm. Clearly,
if a reduction exists, it gives us a polynomial time algorithm for L1 if
L2 ∈ NP :
▶ Lemma (New polynomial-time algorithms via reductions). If
L1 , L2 ∈ Σ∗ are languages such that L1 ≤P L2 , then L2 ∈ P implies L1 ∈ P.
▶ Definition (NP-hard ). A language L ⊆ Σ∗ is NP-hard if
L′ ≤P L for every L′ ∈ NP.
▶ Definition (NP-complete). A language L ⊆ Σ∗ is NP-complete (in NPC)
if L ∈ NP, and L is NP-hard.
▶ Theorem (A first NP-complete problem). CNF-SAT is NP-complete.
▶ Proved by encoding NTMs into SAT Coo71; Lev73 and then CNF-SAT
Kar72. 19/ 34
Strategy for proving NP-completeness
▶ NP-completeness is about hardness: suppose there’s no polynomial-time
algorithm for L1 , and a polynomial-time reduces to L2 ; then, there is no
polynomial-time procedure for L2 .
▶ Lemma. If L is a language such that L′ ≤P L for some L′ ∈ NPC, then L is
NP-hard. If, in addition, L ∈ NP, then L ∈ NPC.
▶ Proof. For all L′′ ∈ NP, we have L′′ ≤P L′ ≤P L. By transitivity, we have
L′′ ≤P L. Thus, L is NP-hard.
▶ Method to prove that a language L is NP-complete: The standard approach
to showing a problem is NP-complete is to polynomial-time reduce a known
NP-complete problem to it. (Also need to show the problem is in NP but
that’s usually (but not always) very easy.)
1. Prove L ∈ NP
2. Prove L is NP-hard.
▶ Select a known NPC language L′ . Then describe an algorithm that computes
a function f mapping every instance x ∈ Σ∗ of L′ to an instance f (x ) of L.
▶ Prove that x ∈ L′ ⇔ f (x ) ∈ L for all x ∈ Σ∗ .
▶ 20/ 34
4. Some popular NP-complete problems

21/ 34
NP-complete problems: 3-CNF SAT is NP-hard

▶ Theorem. 3-CNF SAT is NP-complete.

▶ Proof. 3-CNF SAT is in NP since it is a special case of CNF-SAT. To show
that 3-CNF SAT is NP-hard, we give a polynomial reduction from
CNF-SAT. Let F be a CNF formula. The reduction algorithm constructs a 3
-CNF formula F ′ as follows. For each clause C in F :
▶ If C has at most 3 literals, then copy C into F ′ .
▶ Otherwise, denote C = (ℓ1 ∨ ℓ2 ∨ · · · ∨ ℓk ). Create k − 3 new variables
y1 , . . . , yk−3 , and add the clauses
(ℓ1 ∨ ℓ2 ∨ y1 ) , (¬y1 ∨ ℓ3 ∨ y2 ) , (¬y2 ∨ ℓ4 ∨ y3 ) , . . . , (¬yk−3 ∨ ℓk−1 ∨ ℓk ).
▶ Show that F is satisfiable ⇔ F ′ is satisfiable. Show that F ′ can be
computed in polynomial time (trivial; use a RAM).

22/ 34
Clique and NP-Completeness

▶ Definition. A clique in a graph G = (V , E ) is a subset of vertices S ⊆ V

such that every two vertices of S are adjacent in G.

▶ Clique Problem:
Input: Graph G, integer k
Question: Does G have a clique of size k?
▶ Theorem. Clique is NP-complete.

23/ 34
Proving Clique is in NP
▶ Clique is in NP (HW3!)
▶ For proving Clique is NP-hard, let F = C1 ∧ C2 ∧ . . . Ck be a 3-CNF formula

▶ Construct a graph G that has a

clique of size k iff F is satisfiable

▶ For each clause

Cr = (ℓr1 ∨ · · · ∨ ℓrw ) , 1 ≤ r ≤ k,
create w new vertices v1r , . . . , vwr

▶ Add an edge between vir and vjs if

(i) r ̸= s, and Figure: Graph G corresponding to

(ii) ℓri ̸= ¬ℓsj , where ¬¬x = x . (¬x ∨ y ∨ z) ∧ (x ∨ ¬y ∨ ¬z) ∧ (x ∨ y )

24/ 34
Proving Clique is in NP (cont.)

▶ Check correctness and polynomial running time

▶ Correctness: F has a satisfying assignment iff G has a clique of size k.
▶ (⇒) : Let α be a sat. assignment for F . For each clause Cr , choose a literal
ℓri with α (ℓri ) = 1, and denote by s r the corresponding vertex in G. Now,
{s r : 1 ≤ r ≤ k} is a clique of size k in G since α(x ) ̸= α(¬x ).
▶ (⇐) : Let S be a clique of size k in G. Then, S contains exactly one vertex
sr ∈ {v1r , . . . , vwr } for each r ∈ {1, . . . , k}. Denote by l r the corresponding
literal. Now, for any r , r ′ , it is not the case that lr = ¬lr ′ . Therefore, there
is an assignment α to var(F ) such that α (lr ) = 1 for each r ∈ {1, . . . , k}
and α satisfies F .

25/ 34
Vertex Cover

▶ Definition. A vertex cover in a graph G = (V , E ) is a subset of vertices

S ⊆ V such that every edge of G has an endpoint in S.
Vertex Cover Problem :
▶ Input: Graph G, integer k
Question: Does G have a vertex cover of size k?
▶ Theorem. VERTEX COVER is NP-complete.
▶ Proof. HW3.

26/ 34
Hamiltonian Cycle

▶ Definition. A Hamiltonian Cycle in a graph G = (V , E ) is a cycle visiting

each vertex exactly once.
(Alternatively, a permutation of V such that every two consecutive vertices
are adjacent and the first and last vertex in the permutation are adjacent.)
Hamiltonian Cycle Problem :
▶ Input: Graph G, integer k
Question: Does G have a Hamiltonian Cycle?
▶ Theorem. Hamiltonian Cycle is NP-complete.
▶ Proof. HW3.

27/ 34
Sketch Proof of Hamiltonian Cycle is in NP-Complete

▶ Hamiltonian Cycle is in NP: the certificate is a Hamiltonian Cycle of G.

▶ Let us show: Vertex Cover ≤P Hamiltonian Cycle
▶ Let (G = (V , E ), k) be an instance for Vertex Cover (VC).
▶ We will construct an equivalent instance G ′ for Hamiltonian Cycle (HC).
▶ Intuition: Non-deterministic choices
▶ for VC: which vertices to select in the vertex cover
▶ for HC: which route the cycle takes
▶ Add k vertices s1 , . . . , sk to G ′ (selector vertices)

28/ 34
Sketch Proof of Hamiltonian CyCle is in NP-Complete
(cont.)

▶ Each edge of G will be represented by a gadget (subgraph) of G ′ - s.t. the

set of edges covered by a vertex x in G corresponds to a partial cycle going
through all gadgets of G ′ representing these edges.
▶ Attention: we need to allow for an edge to be covered by both endpoints
▶ Gadget representing the edge {u, v } ∈ E . Its states: ‘covered by u’, ‘covered
by u and v ’, ‘covered by v ’.

29/ 34
The Gadgets

Figure: The gadget, with individual vertices labeled.

30/ 34
The Gadgets (cont.)

(a) (b) (c)

Figure: The gadget used in reducing the vertex-cover problem to the Hamiltonian-cycle
problem. An edge {u; v } of graph G corresponds to gadget Γuv in the graph G ′
created in the reduction. The paths highlighted in blue in Figs. 3a-3c are the only
possible ones through the gadget that include all vertices, assuming that the only
connections from the gadget to the remainder of G ′ are through vertices
[u; v ; 1], [u; v ; 6], [v ; u; 1], and [v ; u; 6].
31/ 34
The Gadgets (cont.): Reducing an instance of VC to an instance
of the HC problem.

Figure: An undirected graph G with a vertex cover of size 2, consisting of the blue
vertices w and y .

Figure: The undirected graph G ′ produced by the reduction, with the Hamiltonian
cycle corresponding to the vertex cover highlighted in blue. The vertex cover (w ; y )
corresponds to edges (s1 ; [w ; x ; 1]) and (s2 ; [y ; x ; 1]) appearing in the Hamiltonian
32/ 34
5. Further Reading and Reeferences

33/ 34
Further Reading and ereferences
▶ Further Reading:
▶ CLRS22’s Chapter 34, NP-Completeness and
▶ Garey and Johnson’s influential reference book, GJ79
▶ References:
[CLRS22] Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford
Stein. Introduction to Algorithms. 4th ed. The MIT Press, 2022.
[GJ79] Michael R. Garey and David S. Johnson. Computers and Intractability: A
Guide to the Theory of NP-Completeness. W. H. Freeman & Co., 1979.
[Coo71] Stephen A. Cook. ”The Complexity of Theorem-Proving Procedures”. In:
Proceedings of the 3rd Annual ACM Symposium on Theory of Computing
(STOC 1971). 1971, pp. 151-158.
[Kar72] Richard M. Karp. ”Reducibility among combinatorial problems.” In:
Complexity of computer computations (Proc. Sympos., IBM Thomas J.
Watson Res. Center, Yorktown Heights, N.Y., 1972). New York: Plenum,
1972, pp. 85-103.
[Lev73] Leonid Levin. ”Universal sequential search problems.” In: Problems of
34/ 34