CAS CS 535: Complexity Theory

Lecturer: Mark Bun Fall 2023

Lecture Notes 4:

Finish NP-Completeness, Decision vs. Search, Time Hierarchy

Reading.

• Arora-Barak § 2.4-2.5, 3.1-3.2, 4.1.3

Last time: Reductions, NP-completeness, Cook-Levin Theorem

Definition 1. Language A is poly-time reducible (a.k.a. Karp-reducible, many-to-one reducible) to B if

there exists a poly-time computable function f : {0, 1}∗ → {0, 1}∗ such that

x ∈ A ⇐⇒ f (x) ∈ B.

Definition 2. A language B is

• NP-hard if A ≤p B for every language A ∈ NP, and

• NP-complete if B is NP-hard and B ∈ NP.

NP-hard
NP-hard

NP-complete

NP P = NP
P = NP-complete

P ≠ NP P = NP

1 An NP-Completeness Example
In the hitting set HS problem, you are given a “ground set” U , natural number k, and sets S1 , . . . , Sℓ ⊆ U .
Your goal is to determine whether there exists a set T ⊆ U such that |T | ≤ k and T ∩ Si ̸= ∅ for every
i ∈ [ℓ].
This is in NP: The certificate is the description of the set T . To verify it, just check that |T | ≤ k and
that T indeed intersects every set Si .

1
 To show that it’s NP-complete, we can reduce from 3SAT. Let φ = (ℓ11 ∨ ℓ12 ∨ ℓ13 ) ∧ · · · ∧ (ℓm m m
1 ∨ ℓ2 ∨ ℓ3 )
be a 3SAT instance on variables x1 , . . . , xn . Our reduction computes the following HS instance:

U = {x1 , x1 , . . . , xn , xn }
S1 = {ℓ11 , ℓ12 , ℓ13 }
...
Sm = {ℓm m m
1 , ℓ2 , ℓ3 }
Sm+1 = {x1 , x1 }
...
Sm+n = {xn , xn }
k = n.

This reduction takes time polynomial in the instance size. To show that it is correct, we need to show that φ
is satisfiable ⇐⇒ ⟨U, S1 , . . . , Sm+n , k⟩ ∈ HS.

⇒: Let (b1 , . . . , bn ) be a satisfying assignment to φ. We construct a hitting set T as follows. For each
i = 1, . . . , n, if bi = 1, place xi in T . If bi = 0, place xi in T . This gives a hitting set of size n = k.

⇐: Let T be a hitting set of size k = n. Since T must intersect Sm+1 , . . . , Sm+n , it must contain exactly
one element of each pair. Construct the satisfying assignment (b1 , . . . , bn ) by taking bi = 1 ⇐⇒ xi ∈ T .

2 Decision vs. Search

For much of this class, we focus on decision problems (languages), i.e., those with yes/no answers. Why do
we do this when computation encompasses so many other types of problems?

1. This is the simplest setting for studying complexity, where we already can’t answer most of the inter-
esting questions we want to ask.

2. Solving decision problems actually lets us solve other types of computational problems!

Today, we’ll address the second point by describing about a generic “search-to-decision” reduction for
NP. But to set up this general reduction, let’s start by looking at a concrete problem.
Recall the decision problem

SAT = {φ | φ has a satisfying assignment}.

This corresponds to a natural search problem:

SAT − Search: Given a CNF φ, output a satisfying assignment to φ if one exists, and ⊥ otherwise.
Theorem 3. If SAT ∈ P, then there exists a poly-time algorithm solving SAT − Search.
Proof. Let A be a poly-time algorithm deciding SAT. The idea is to repeatedly apply A to identify a
satisfying assignment bit-by-bit.
First some notation: For a formula φ and b ∈ {0, 1}, let φ|x1 =b be the formula φ with variable x1
replaced by the constant b. (And similarly if we want to fix more variables)

2
 Observe that:
φ ∈ SAT ⇐⇒ φx1 =0 ∈ SAT or φx1 =1 ∈ SAT.
The following procedure uses A to construct a satisfying assignment b1 , . . . , bn to a formula φ(x1 , . . . , xn ).
Alg Find(φ):
If A(φ) = 0: Output ⊥.
For i = 1, . . . , n:
If A(φ|x1 =b1 ,...,xi−1 =bi−1 ,xi =0 ) = 1: // i.e., φ|x1 =b1 ,...,xi =0 (xi+1 , . . . , xn ) ∈ SAT
Set bi = 0
Else:
Set bi = 1
Output (b1 , . . . , bn )
This algorithm uses n + 1 invocations of the poly-time algorithm A, so it runs in poly-time.

This result is just an example of a more general phenomenon. To state this, we need the following
definition of an NP search problem.

Definition 4. Let f : {0, 1}∗ × {0, 1}∗ → {0, 1} be a poly-time computable function (“verification func-
tion”), and let p : N → N be a polynomial (“certificate length”). The NP search problem associated to f
and p is as follows.
Given x ∈ {0, 1}∗ , find a u ∈ {0, 1}p(|x|) such that f (x, u) = 1 (if one exists).

For example, we can capture the SAT − Search problem by taking f (φ, u) = 1 ⇐⇒ u satisfies the
CNF φ.
As another example, we can take f (⟨G, k⟩, u) = 1 ⇐⇒ u is an independent set for G of size k.

Theorem 5. If P = NP, then every NP search problem can be solved in poly time.

Proof. Consider an NP search problem defined by verification function f and certificate length p. The idea
will, again, be to construct a certificate bit-by-bit using our ability to solve a decision problem associated to
f in poly-time.
One choice of decision problem that works here is

Lf = {⟨x, b⟩ | ∃z ∈ {0, 1}p(|x|)−|b| s.t. f (x, b ◦ z) = 1}.

That is, Lf consists of pairs (x, b) such that there exists a way to extend the prefix b to a certificate u = b ◦ z
for which f (x, u) = 1.
The language Lf ∈ NP because f is poly-time computable. Therefore, if P = NP, there is a poly-time
algorithm A deciding Lf . This gives us the following search-to-decision reduction:
Alg Find(x):
If A(x) = 0: Output ⊥.
For i = 1, . . . , p(|x|):
If A(x, b1 , . . . , bi−1 , 0) = 1: // i.e., ⟨x, b1 , . . . , bi−1 , 0⟩ ∈ Lf
Set bi = 0
Else:
Set bi = 1
Output (b1 , . . . , bp(|x|) ).

3
3 Hierarchy Theorems
Reductions and the theory of NP-completeness give us tools to understand the relative hardness of compu-
tational problems. But on their own, they don’t give us a way to establish, say, that any of the problems in
NP are themselves actually hard.
The main tool we have for actually separating complexity classes is diagonalization, which we’ll first
use here to prove hierarchy theorems. A hierarchy theorem says that strictly more of a particular resource
lets you compute strictly more things.
We’ll start with the deterministic time hierarchy theorem, where the resource is time on a deterministic
TM. To state the theorem precisely, we need a slightly annoying definition.

Definition 6. A function f : N → N is time-constructible if there exists a TM computing f (|x|) within

O(f (|x|)) steps.

Most reasonable functions f (n) ≥ n are time-constructible, e.g., n1.5 , n log n, 10n . Some functions that
are not time-constructible are log n, n0.9 (too small), Busy-Beaver(n) (too large to be computable).

Theorem 7 (Deterministic Time Hierarchy Theorem). If f and g are time-constructible and f (n) log f (n) =
o(g(n)), then
DTIME(f (n)) ⊊ DTIME(g(n)).

That is, while DTIME(f (n)) ⊆ DTIME(g(n)) (obviously), there exists a language L ∈ DTIME(g(n)),
but L ∈/ DTIME(f (n)).
For example, if f (n) = n and g(n) = n log2 n, then because n log n = o(n log2 n) we have that
DTIME(n) ⊊ DTIME(n log2 n).
Before proving the theorem, I want to draw attention to two apparent deficiencies. One is the “extra”
factor of log f (n); ideally, one would like to be able to separate time classes whenever f (n) = o(g(n)).
This factor has to do with the best known simulation of arbitrary-tape TMs by 3-tape TMs, which incurs the
same logarithmic runtime blowup.
Second is the restriction to time-constructible f and g. This looks like a technicality, but it turns
out to be important to the truth of the statement. There’s a result called the Borodin-Trakhtenbrot Gap
Theorem which, as a consequence, shows that there is a non-time constructible function f for which
DTIME(f (n)) = DTIME(2f (n) ).
As mentioned before, the idea we’ll use to prove the time hierarchy theorem is diagonalization. This is
the same idea that was implicit in our construction of the undecidable language SNA, so let’s spell out that
construction in a bit more detail.
Imagine constructing a 2-dimensional grid where rows are indexed by possible input strings x, and
columns are indexed by Turing machines encoded by strings α. In each cell (x, α), we write the value of
Mα (x).

4
 (Encodings of) Turing machines

𝛼1 𝛼2 𝛼3 𝛼4
𝑥1 0 1 1 0 0 …
𝑥2 1 1 0 0 000
Inputs

𝑥3 11 10 ⊥ 1 0
𝑥4 1 0 1 1001 1
…

The language SNA was defined by “flipping the diagonal” to force every TM Mα to make a mistake
when run on input x = α. That is,
SNA = {α | Mα (x) ̸= 1}.
To prove our time hierarchy theorem, we’d like to use the same idea, but

1. The language we obtain by flipping the diagonal should be decidable in O(g(n)) time

2. We only need to thwart machines that run in time O(f (n)).

As a first attempt to do this, we might try only enumerating over all TMs running in time O(f (n)). The
problem is that it’s undecidable to determine whether a given TM M runs in time O(f (n)). So instead,
we’ll go back to enumerating over all TMs, but stop each simulation early, after only g(n) steps.

Proof. We construct a language L ∈ DTIME(g(n)) \ DTIME(f (n)) as follows. Construct the TM

D(x) =

1. Run the UTM U (x, x) to simulate Mx on input x, and stop after g(|x|) time steps.

2. If U has output 1, return 0. Otherwise, return 1.

Let L be the language decided by D. By construction, D ∈ DTIME(g(n)).

Now suppose for the sake of contradiction that some TM M decides L in time O(f (n)). By the effi-
ciency of the UTM, simulating M to completion on any input x takes time cM f (|x|) · log f (|x|) for some
cM depending only on the machine M .
Now this is the part of the proof where some assumptions about how we encode TMs matters. Let us
assume that every TM has infinitely many encodings (say, by encoding an arbitrary number of trailing 0s).
Then the fact that f (n) log f (n) = o(g(n)) implies that there exists some encoding x∗ of M such that
g(|x∗ |) > cM f (|x∗ |) · log f (|x∗ |).
Thus, when we run D(x∗ ), the simulation has enough time to finish, and so D(x∗ ) = 1 − U (x∗ , x∗ ) =
1 − M (x∗ ), a contradiction.

Next time: Nondeterministic time hierarchy, Ladner’s theorem, limits of diagonalization.