COMS W3261: Computer Science Theory

Omer Mustel, Alex Lindenbaum (with huge credits to Rahul Gosain and Benjamin Kuykendall,
2018 TAs)

Handout 11: Final Review

Key Terms
Set: a group of objects, which we call its elements. These “objects” can be almost anything:
symbols, numbers, strings, or sets themselves. We have multiple ways to define sets:

Enumerate the items of a finite set: e.g. {1}, {1, 2, 3}, {a, b, c}.
Use an ellipsis to continue a pattern infinitely: e.g. {1, 2, 3, . . . }, {1, 11, 111, . . . }.
Give a condition for membership: e.g. {n | n = m2 for some m ∈ N}.

We have symbols for members and subsets:

Membership: say x ∈ S (or “x is in S”) if x is an element of S.

Subset: say T ⊆ S (or “T is a subset of S”) if every element in T is also in S.
Powerset: let P (X) (or “the set of all subsets of X”) be P (X) = {S | S ⊆ X}.
Note that ∅ ∈ P (X) and X ∈ P (X).

We have a few basic operations on sets. e.g.

Union: A ∪ B = {x | x ∈ A or x ∈ B}.
Intersection: A ∩ B = {x | x ∈ A and x ∈ B}.
Difference: A\B = {x | x ∈ A and x 6∈ B}.

Alphabet: a non-empty finite set of symbols, often denoted Σ. e.g.

{0, 1}, {0, 1, . . . , 9}, {a, b, c}, {a, b, c, . . . , z}.

String: a finite sequence of symbols from the alphabet. e.g.

ε (the empty string, which has no symbols)

1, 11, 111, 1996, benjamin

Encodings: finite mathematical objects can be encoded in a string over any language, e.g.

Integers (using their base-n representations)

Lists and sets (by separating the objects by commas)
Graphs (by listing the vertices and edges)
Functions between finite sets (by listing the pairs (x, f (x)))

Further, since all of our models of computation are tuples of sets and functions, we can
encode DFAs, NFAs, PDAs, and TMs. If x is an object, denote the encoding by hxi.
In general, not all strings will be valid encodings. For example, if we are encoding pairs of
integers in the format x#y, then 123#321 is a valid encoding, but ## is not.

1
Language: a set of strings from the same alphabet. e.g.

∅ (the set of no strings)

Σ∗ (the set of all strings over Σ)
A = {w ∈ {0, 1}∗ | w contains a 1}
GC = {hGi | G is a cyclic graph}
M0110 = {hM i | M is a TM that accepts 0110}

Complement of language: the complement L is the set of strings in Σ∗ but not in L.

Or in set theory notation L := Σ∗ \L. e.g.

∅ = Σ∗
Σ∗ = ∅
A = {0n | n ≥ 0}
GC = {hGi | G is not a graph or G is an acyclic graph}
M0110 = {hM i | M is not a TM or M is a TM that does not accept 0110}

Turing Machine: we will formally define a Turing machine later; for now, consider the
three possible results when you run a Turing machine M on a string w:

After some number of steps, the machine enters the accept state: say “M accepts w.”
After some number of steps, the machine enters the reject state: say “M rejects w.”
The machine reaches neither the accept nor the reject state: say “M loops on w.”

We say “M halts on w” if it either accepts or rejects.

Recognizer: “M recognizes L” when

w ∈ L ⇐⇒ M accepts w.

The language recognized by M is unique, namely L(M ) = {w ∈ Σ∗ | M accepts w}.

Decider: “M decides L” when both

w ∈ L =⇒ M accepts w and w 6∈ L =⇒ M rejects w.

Or equivalently:
If M recognizes L and M halts on all inputs.
Again, if M decides any language, that language is L(M ).

Computable function: a function f : Σ∗ → Σ∗ is computable when there is some Turing

machine (technically an input-output Turing machine) that, on input string x, will contain
f (x) on the tape when it halts.

Set of Languages: like any mathematical objects, we can have a set of languages. e.g.

{A} (the set containing only the language A)

{L | L ⊆ {0, 1}} (the four languages ∅, {0}, {1}, and {0, 1})
{L | L ⊆ Σ∗ and L is recognizable} (or regular, decidable, unrecognizable . . . )
{L | L ⊆ Σ∗ and L can be decided in polynomial time} (the class P)
{L | L ⊆ Σ∗ and L can be verified in polynomial time} (the class NP)

2
 It is important to be able to distinguish different types of sets: sets of strings (languages)
versus sets of languages. Here are some tricky examples of sets of languages that you should
understand. e.g.
{∅} (the set containing only the language ∅)
∅ (the set containing no languages)
{L | L ⊆ Σ∗ } (the set of all languages over Σ∗ )
{Σ∗ } (the set containing only the language Σ∗ )

Set Cardinality
The cardinality of a set is its size, denoted by |S|. For a finite set, |S| is simply the number of
elements. Infinite sets have cardinalities as well. To show that two sets have the same cardinality,
i.e. |S| = |T |, we need a bijection f from S to T (recall that f is a bijection iff for each y ∈ T ,
there is a unique x ∈ S such that f (x) = y.) For finite sets, it should be clear why a bijection
says the sets must have the same number of elements. For infinite sets, you can take this to be
a definition.
Proof Template 1: to show |S| ≤ |T |

1. Construct a function f from S to T .

2. Check that f maps distinct elements of S to distinct elements of T .

3. Conclude that f is an injective function, so |S| ≤ |T |.

Example 1:

Let Z = {. . . , −2, −1, 0, 1, 2, . . . } (the integers) and Z≥0 = {0, 1, 2, 3, . . . } (the non-negative
integers). We claim that |Z| = |Z≥0 |. Follow the proof template above: construct a function
f : Z → Z≥0 : (
2x if x ≥ 0
f (x) =
1 − 2x if x < 0.
Consider two elements x, y ∈ Z. If f (x) = f (y) and the value is even, then we know x =
f (x)/2 = f (y)/2 = y. Otherwise, if it odd, we know x = −(f (x)−1)/2 = −(f (y)−1)/2 = y.
So conclude f (x) = f (y) iff x = y. Thus f is injective, giving |Z| ≤ |Z≥0 |.

Further, since Z≥0 ⊆ Z, we know |Z| ≥ |Z≥0 |. Conclude that |Z| = |Z≥0 |.

This same proof template can be used in two important ways. Consider a set S. Then
To prove S is countable, show that |S| ≤ |N| (or some other countable set).
To prove S is uncountable, show that |S| ≥ |P (N) | (or some other uncountable set).
We have some adjectives to describe sets with certain cardinalities: an “inifinite” set is either
countably or uncountably infinite, a “countable” set is either finite or countably infinite, and an
“uncountable” set is uncountably infinite.

3
 Countable Infinite

{1, 2} N P (N)

Finite Countably infinite Uncountable

∅ Σ∗ R
Q Z P (Σ∗ )
Σ

Figure 1: Relationships between different types of cardinalities.

Cantor’s diagonalization proves |N| < |P (N) |. (In fact, for any infinite set X, we know P (X)
is uncountable). In other words, countable sets are smaller than uncountable ones. From this
we know that there is no surjective map from N to P (N).

Application to Recognizability
Let M be the set of all Turing machines. Consider the encoding map f (M ) = hM i. Since
encoding is injective from M to Σ∗ , and Σ∗ is countable, this gives that M is countable. Let us
continue to describe sets we know from our study of Turing machines in terms of cardinality:

Σ: finite.
Σ∗ : countably infinite.
L ∈ Σ∗ (any language): countable.
M (the set of Turing machines): countably infinite.
P (Σ∗ ) (the set of all languages): uncountable.

Now consider the function from a Turing machine to the language it recognizes L : M →
P (Σ∗ ). Since P (Σ∗ ) is uncountable, we know the map cannot be surjective. In other words,
there is some language S ∈ P (Σ∗ ) such that no Turing machine recognizes S. This same proof
technique can be used to show that any uncountable set of languages contains some unrecogniz-
able language.

4
Turing Machines
Formal Definition
A Turing machine is a 7-tuple (Q, Σ, Γ, δ, q0 , qaccept , qreject ) where
1. Q is a finite set of states,

2. Σ is the input alphabet, 6∈ Σ,

3. Γ is the tape alphabet, ∈ Γ and Σ ⊂ Γ,

4. δ : Q × Γ → Q × Γ × {L, R} is the transition function,

5. q0 ∈ Q is the start state,

6. qaccept ∈ Q is the accept state,

7. qreject ∈ Q is the reject state, qaccept 6= qreject .

This may seem like a lot to deal with. But once you understand how the transition function
works, each line of this definition should make sense. The machine operates by reading the
character γ at the head, computing δ(q, γ) = (q 0 , γ 0 , D), writing γ 0 on the tape at the current
position, moving the head left or right depending on D, and entering state q 0 . It repeats, stopping
only if one of qaccept or qreject is reached.

High Level Descriptions

We use Turing machines and algorithms interchangeably. In class, we showed that Turing ma-
chines are able to do many operations involving strings and numbers; in fact, they are much
more powerful. Any algorithm you could write in a programming language can be simulated by
a Turing machine: the Church-Turing thesis postulates that Turing machines can simulate any
reasonable model of computation. Thus, when giving a Turing machine specification, we often
use high level descriptions (i.e. psuedocode) to describe a machine.

Examples of permissible syntax and operations include:

ˆ Variables: assign variables and use their values, add to or remove from finite lists and sets,
check membership in a finite list or set.

ˆ Control flow: use loops, jump conditionally, or have if-then-else statements.

ˆ String operations: move strings, append them to one another, reverse them, find the nth
character of a string, find and replacing substrings.

ˆ Arithmetic operations: add, subtract, multiply, divide, or exponentiate integers or rational

numbers, check if one number divides another, compare or check equality.

ˆ Graphs: check if graph encodings are valid, check if a vertex has neighbors, add or remove
edges and vertices, place marks on edges and vertices.

ˆ Turing machines: check if an encoding is valid, run a Turing machine on an input, run a
Turing machine on an input for some number of steps, write a new Turing machine.

5
 The operations on Turing machines can be subtle. Because Turing machines can run forever
(and there is no way to check if a Turing machine will run forever, as we proved HALTTM is
undecidable) we have to account for this in our constructions.

Example 2:

Let M be a Turing machine. We will construct another Turing machine as follows.

N = “On input w,

1. Simulate M on w.
2. If M accepts, reject.
3. If M rejects, accept.”

If M decides L, then we know N decides L. To prove correctness:

w 6∈ L =⇒ M rejects w =⇒ N accepts w.

w ∈ L =⇒ M accepts w =⇒ N rejects w.

However, if M only recognizes L, it is possible that N could loop forever in step 1. For
example, consider the case where M never rejects: instead, whenever it encounters a input
w 6∈ L, it loops forever. In that case:

w 6∈ L =⇒ M loops on w =⇒ N loops on w.

w ∈ L =⇒ M accepts w =⇒ N rejects w.

Thus, in this case, N accepts no strings; in other words, it recognizes ∅.

Another place we have to account for this effect is in the recognizer for ATM . There we either
use a technique called dovetailing to avoid looping, or take advantage of NTMs. See more details
in lecture 18 and handout 8.

Example
Consider the language
L = {w ∈ {0, 1}∗ | w contains no 1’s}.
We will give four Turing machines, in different formats, that all recognize L. This is intended to
show you the wide range of formats that are acceptable for describing algorithms.

High level verbal description

M = “On input x ∈ {0, 1}∗ , check if x contains any 1’s. If so, reject, else accept”

6
Formal description
This high level description corresponds to the following TM (described in a formal level):

M = ({q0 , qaccept , qreject }, {0, 1}, {0, 1, }, δ, q0 , qaccept , qreject )

where δ is defined by

δ(q0 , 0) = (q0 , 0, R), δ(q0 , 1) = (qreject , 1, R), δ(q0 , ) = (qaccept , , L)

Or, the same TM written as a state diagram:

0 → 0, R

→ ,L
start q0 qaccept

1 → 1, R

qreject

Real world programming language

public boolean isInLanguage ( String x ) {

for ( int i = 0; i < x . length (); i ++) {
if ( x . charAt ( i ) != ’0 ’) {
return false ;
}
}
return true ;
}
Pseudocode (unnatural algorithm)
The above algorithms were straight-forward and natural. However, note that there are (infinitely
many) other algorithms that recognize the same language. Here is one of them.

M = “On input x ∈ {0, 1}∗ ,

1. If x starts with 1, loop forever.

2. Check if x contains any 1’s, and if so reject.
3. Calculate the 7th digit of π and if it is even accept, else reject”

Although you would never write such a convoluted algorithm yourself, you should remember
that recognizers do not need to be reasonable: if you are given a recognizer in the problem, it
could very well be doing “silly” operations like this under the hood.

7
Reductions
In general, a reduction is a way to order problems based on how hard they are. Formally, we
say A ≤ B or “A is reducible to B”. Here are three English-language interpretations of that
statement that give good intution about reductions:
A ≤ B ⇐⇒ “the problem of solving A reduces to the problem of solving B”
⇐⇒ “if we can solve B, then we can solve A”
⇐⇒ “A is easier than [or just as easy as] B”.
Reduction come in multiple flavors: we studied ≤T , ≤m , and ≤p .
Turing reduction say A ≤T B (or “A is Turing-reducible to B”) when there is some
algorithm N , which has access to an oracle M and
M decides B =⇒ N decides A.

Mapping reduction say A ≤m B (or “A is mapping-reducible to B”) when some com-

putable function f is such that
x ∈ A ⇐⇒ f (x) ∈ B.

We know that given a mapping reduction, there is also a Turing reduction. Thus wherever
you might need a Turing reduction, you could use a mapping reduction instead. However, the
converse does not hold: in some cases there is a Turing reduction, but no mapping reduction. In
the next section, we will go over how both types of reductions can be used to solve problems.

Proof Techniques
In this section, you’ll see that there are three techniques that you should consider when you
approach a problem: direct demonstrations, reductions or Rice’s theorem. Not all will work on
every problems, but the proof templates below should help you get the hang of it.
We note that in all examples we explicitly check whether the input is a valid encoding, and
deal with it appropriately. However, you may use the convention we often used in class of
omitting this check (assuming that it’s implicitly there, since it’s easy to deal with).

Recognizability
Recall that a language L is recognizable if we can construct a Turing machine that accepts only
all the strings in L. That is, it accepts every string in L and rejects or loops infinitely on all
strings that are not in L. There are two ways you can prove a language is recognizable.
Proof Template 2: to show L is recognizable via a recognizer

1. Write a description of a Turing machine M that recognizes L.

2. Assuming x ∈ L, prove that M accepts x.

3. Assuming x 6∈ L, prove that M does not accept x.

4. Conclude that since M recognizes L, L is recognizable.

8
 Example 3:

Recall the language

ETM = {hM i | M is a TM and L(M ) = ∅}.

We will show the complement ETM is recognizable by constructing a recognizer.

MN = “On input x

1. Check that x = hM i, an encoding of a TM. If not, accept.

2. Let i = 1.
3. Run M on every string of length at most i for at most i steps each.
If any simulation sees M accept, then accept.
4. Increment i and return to step 2.”

Now we will prove correctness:

Assume x ∈ E T M . Then either x 6= hM i or x = hM i where M is a TM with L(M ) 6= ∅.

In the first case, then x is accepted in Step 1. In the second, then there is some w ∈ L(M )
that is accepted in some number of steps c. So, if w is not already accepted, then it will
definitely be accepted by MN in step 2 when i = max(c, |w|). In either case MN will accept x.

Assume x 6∈ E T M . Then x = hM i where M is a TM with L(M ) = ∅. This means that

x will pass the check in Step 1, and every simulation of M in Step 3 will not result in an
accept. So, MN will run indefinitely and will not accept x.

Conclude that since MN is a recognizer for ETM , that ETM is recognizable.

A second approach to showing recognizability is by showing a mapping reduction to another

language you already know is recognizable.
Proof Template 3: to show L is recognizable via mapping reduction

1. Pick a recognizable language B. Write an algorithm that computes f .

2. Assuming x ∈ L, prove that f (x) ∈ B.

3. Assuming x 6∈ L, prove that f (x) 6∈ B.

4. Conclude that since L ≤m B and B is recognizable, L must be too.

Decidability
Decidability is a lot like recognizability, except we impose an extra condition on the Turing
machine: it must reject all strings that are not in L. That is, a decider always gives us back an
answer in finite time for all possible inputs. We can use the same proof strategy we used above
and supply the decider explicitly, or we can use a reduction.

9
 Proof Template 4: to show L is decidable via a decider

1. Write a description of a Turing machine M that is a decider for L.

2. Assuming x ∈ L, prove that M accepts x.

3. Assuming x 6∈ L, prove that M rejects x.

4. Conclude that since M is a decider for L, L is decidable.

Example 4:

Recall the language

SAT = {hφi : φ is a Boolean formula with a satisfying assignment}.

We claim that SAT is decidable. We will define a Turing machine.

MSAT =“On input x

1. Check that x = hφi, an encoding of a Boolean formula of some number of variables

n. If not, reject.
2. Iterate through all 2n possible truth assignments to the variables in φ and evaluate
φ with each. If one makes φ evaluate to True, accept.
3. After iterating through all assignments (without finding a satisfying assignment),
then reject.

Now we will prove correctness:

Assume x ∈ SAT. Then x = hφi where φ is a Boolean formula with a satisfying assignment.
This means that x will pass the check in Step 1. Then, x will be accepted in Step 2 since
there is some satisfying assignment that will be checked and cause φ to evaluate to True.
So, MSAT will accept x.

Assume x 6∈ SAT. Then either x 6= hφi for any Boolean formula φ or x = hφi where φ is a
Boolean formula with no satisfying assignment. In the first case, x will be rejected in Step
1. In the second, every assignment tested in Step 2 will not evaluate to True (since there is
no satisfying assignment). So, MSAT will reach Step 3 and reject x.

Conclude that since MSAT is a decider for SAT, that SAT is decidable.

Note that a proof similar to this one can be given to show that in fact every langauge in NP is decidable.

You can also show decidability by using a Turing reduction, utilizing a decider for another
langauge you already know how to decide:

10
 Proof Template 5: to show L is decidable via Turing reduction

1. Pick some decidable language B and assume you have a decider MB for it.

2. Demonstrate that you can use MB to create a decider ML for L.

3. Assuming x ∈ L, prove that ML accepts x.

4. Assuming x 6∈ L, prove that ML rejects x.

5. Conclude that since L ≤T B and B is decidable, L must be too.

Undecidability
A language L is undecidable if it’s impossible to construct a Turing machine that is a decider
for L. Since we want to make a claim about all Turing machines, we can’t check all the options
individually. Most of the time, the best strategy is to use a reduction to prove that L is “at least
as difficult” as a language.

Proof Template 6: to show L is undecidable via Turing reduction

1. Assume that you have a decider ML for L.

2. Pick an undecidable language U .

3. Demonstrate that you can use ML to create a decider MU for U .

4. Assuming x ∈ U , prove that MU accepts x.

5. Assuming x 6∈ U , prove that MU rejects x.

6. Conclude that since U ≤T L and U is undecidable, L must be undecidable as well.

11
 Example 5:

We want to show ETM is undecidable; we already know that AT M is undecidable via the
diagonalization proof in class. For reference, here are the definitions of the two languages:

ETM = {hM i | M is a TM and L(M ) = ∅}.

AT M = {hM, wi | M is a TM and M accepts w}.

In order to carry out the reduction, assume there exists a decider ME for ETM . We could
use ME to create a decider for AT M as follows.

MA =“On input x,

1. Check if x = hM, wi, an encoding of a TM M and string w. If not, reject.

2. Create an encoding of the following Turing machine:
M 0 =“On input z, run M on w and output the same.”
3. Run ME on input hM 0 i. If ME accepts, then reject and vice versa.”

Now we will prove correctness:

Assume x ∈ AT M . Then x = hM, wi where M is a TM that accepts when run on input w.

This means that the constructed M 0 will accept on all inputs. So, ME will reject when run
on hM 0 i (since it is a decider for ETM ), and MA will therefore accept x.

Assume x 6∈ AT M . Then either x 6= hM, wi or x = hM, wi where M is a TM that does

not accept w. In the first case, x is rejected in Step 1. In the second, the constructed M 0
accepts no strings. Namely, L(M 0 ) = ∅. So, ME will accept when run on hM 0 i (since it is a
decider for ETM ), and MA will therefore reject x.

Conclude that since AT M ≤T ETM and AT M is undecidable, that ETM is undecidable too.

An alternative strategy is to use Rice’s Theorem. Define a property of recognizable languages

to be any set of TM encodings P ⊆ {hM i : M is a TM}, where any two TMs that recognizae
the same language are both in P or both not in P (namely, for any pair of TMs M1 , M2 with
L(M1 ) = L(M2 ) it holds that hM1 i ∈ P ⇐⇒ hM2 i ∈ P ).

Theorem 1. (Rice’s Theorem Let P be a property of recognizable languages, such that P is

non-trivial (namely there exists at least one TM M with hM i ∈ P and at least one TM M 0 with
hM 0 i ∈
/ P ). Then P is not decidable.

12
 Proof Template 7: to show L is undecidable using Rice’s theorem

1. Argue that L is a property of recognizable languages, namely L ⊆ {hM i : M is a TM}

(a set of TMs).

2. Demonstrate one TM whose encoding is in L.

3. Demonstrate one TM whose encoding is not in L.

4. Conclude that by Rice’s theorem, L is undecidable.

Example 6: ETM is not decidable

Recall the language

ETM = {hM i | M is a TM and L(M ) = ∅}.

ETM is clearly a property of recognizable langauges, from its definition.

To show that ETM is non-empty, note that hM∅ i ∈ ETM where M∅ is the TM that immedi-
ately rejects all inputs.

To show that ETM is not all TM encodings, note that hMall i ∈ ETM where Mall is the TM
that immediately accepts all inputs.

Conclude that ETM is a non-trivial property of recognizable languages, so by Rice’s theorem

ETM must be undecidable.

Unrecognizability
A language L is unrecognizable if it’s impossible to construct a Turing machine that is a recognizer
for L. As in the previous section, we’re going to want to use a mapping reduction to make a
claim about all possible Turing machines. The reduction strategy looks very similar.

Proof Template 8: to show L is unrecognizable via mapping reduction

1. Pick an unrecognizable language U . Write an algorithm that computes f .

2. Assuming x ∈ U , prove that f (x) ∈ L.

3. Assuming x 6∈ U , prove that f (x) 6∈ L.

4. Conclude that since U ≤m L and U is unrecognizable, L must be unrecognizable too.

We can also prove unrecognizability by relying on the theorem that a language L is decidable
if and only if both L and its complement L are recognizable. The following strategy works only
for languages that are undecidable, but their complement is recognizable.

13
 Proof Template 9: To show L is unrecognizable by analyzing L and L

1. Prove that L is undecidable (e.g., using Rice’s theorem).

2. Prove that L is recognizable.

3. Conclude that L is therefore un-recognizable (since it it were, then L, L would both

be recognizable, so L would be decidable).

Finally, We could use refined Rice’s theorem for to prove unrecognizability, in a similar way
to the use of Rice’s theorem to prove undecidability. The condition of refined Rice’s theorem
is the same as the one in the standard Rice’s theorem (and the proof is the same too), but the
conclusion is stronger. This is bonus material (in notes of Lecture 21) which is not required for
class, but you are welcome to use it.

Theorem 2. (Refined Rice’s Theorem) Let P be a property of recognizable languages, such that
P is non-trivial. Let hM∅ i be the TM that immediately rejects all inputs.

ˆ If hM∅ i ∈ P then P is not recognizable

ˆ If hM∅ i ∈
/ P then P is not recognizable.

Proof Template 10: to show L is unrecognizable using Refined Rice’s theorem

1. Argue that L is a property of recognizable languages, namely L ⊆ {hM i : M is a TM}

(a set of TMs).

2. Demonstrate one TM whose encoding is in L.a

3. Demonstrate one TM whose encoding is not in L.

4. Demonstrate that hM∅ i ∈ L

5. Conclude that by Refined Rice’s theorem, L is unrecognizable.

a
This also follows from step 4 so could be skipped

14
Complexity
We have a very rich theory of computability saying which languages are decidable, recognizable,
or neither: roughly, whether a problem can be solved by a computer. However, this tells us
nothing about how efficiently it can be solved. Complexity is the study of how much resources
are needed to solve a problem. Efficiency is always measured in terms of the length of the input
(i.e. the number of characters), which we denote by n.

TIME(f (n)): the class of languages decidable by Turing machines that run in time O(f (n)).

NTIME(f (n)): the class of languages decidable by non-deterministic Turing machines that
run in time O(f (n)).

P: the class of languages decidable by Turing machines that run in time O(nk ) for some
constant k. In other words ∞
[
P= TIME(nk ).
k=1

NP: the class of languages decidable by non-deterministic Turing machines that run in
time O(nk ) for some constant k. In other words
∞
[
NP = NTIME(nk ).
k=1

Alternatively, NP is the class of languages with polynomial time verifiers; so A ∈ NP when

there exists a V , a polynomial time verifier such that

x ∈ A ⇐⇒ ∃c such that V accepts hx, ci

Poly-time mapping reduction: a mapping reduction that runs in time O(nk ) for some
constant k, write A ≤p B or “A is polynomial-time mapping-reducible to B”.

NP-Hard: a problem L is NP-hard when for all S ∈ NP, S ≤p L.

NP-Complete: a problem is NP-complete when it is NP-hard and in NP.

Problems in NP

Proof Template 11: to show L is in NP using a verifier

1. Write a verifier V as an algorithm that takes hx, ci.

2. Show that V runs in time O(|x|k ) for some k.

3. Assuming x ∈ L, prove that ∃c such that V (x, c) accepts.

4. Assuming x 6∈ L, prove that ∀c that V (x, c) must reject.

5. Conclude that since V is a polynomial-time verifier for L, that L ∈ NP.

15
 Example 7:

Recall a simple cycle is a cycle with no repeated vertices. Define the following problem:

CycleSize = {hG, ki | G is a graph with a simple cycle of size k}

Claim that CycleSize ∈ NP. To show this, we will construct a verifier.

VC = “On input x, c,

1. Checks that x = hG, ki an encoding of a graph G along with integer k. If not,

reject.
2. Checks that c is a list of k distinct vertices of G. If not, reject.
3. For each i = {1, . . . k}, check that (ci , ci+1 ) is an edge in G. If not, reject.
4. Accept.

First we will analyze runtime: claim VC operates in polynomial time in the size of n = |x|
because encoding validation (steps 1 + 2) can be done in polynomial time and step 3 consists
of at most |V | edge lookups, where |V | ≤ n, so it is a polynomial time.

Now we will prove correctness:

If x ∈ CycleSize, then x = hG, ki where G is a graph with a simple cycle of size k. Let
c be a list of the vertices in G’s simple cycle of size k. When x, c are given to VC , c will
therefore pass the checks in steps 1, 2, and 3. So, VC will accept x, c.

If x 6∈ CycleSize, then either x 6= hG, ki or x = hG, ki where G is a graph with no simple

cycle of size k. In the first case, VC will always reject in step 1. In the second, no matter
what c is input to ML it will not pass steps 2 and 3 (doing so would mean that c is a simple
cycle of length k in G). So, VC will reject x.

Conclude VC is a polynomial-time verifier for CycleSizeand therefore CycleSize ∈ NP.

Proof Template 12: to show L is in NP via an NTM

1. Write an NTM M .

2. Show that M runs in time O(nk ) for some k.

3. Assuming x ∈ L, prove that some branch of M accepts x.

4. Assuming x 6∈ L, prove that no branch of M accepts x.

5. Conclude that since M decides L in time O(nk ), that L ∈ NTIME(nk ) ⊆ NP.

16
NP Hardness

Proof Template 13: to show L is NP-Hard using a poly-time reduction

1. Pick some H that is NP-Hard. Write an algorithm that computes f .

2. Check that your algorithm runs in time O(nk ) for some k.

3. Assuming x ∈ H, prove that f (x) ∈ L.

4. Assuming x 6∈ H, prove that f (x) 6∈ L.

5. Conclude that since H ≤p L and H is NP-hard, that L is NP-hard.

Below is an example of a reduction proving NP-hardness according to the above template. We

note that given how little time we had for complexity this year, you will not be asked to produce
a full proof of NP hardness on the exam. But the example is useful since you are expected to
understand what a reduction is, the definition of NP-hardness, etc.
We also note that as before, here we take care to deal with bad encodings explicitly, but it is
also ok in this class to ignore it and implicitly assume that strings that are not a valid encoding
are dealt with implicitly (for the case of standard encodings that are easy to check).

17
 Example 8:

We claim CycleSize is also NP-hard. To show this, we start with the HamCycle problem

{hGi | G a graph with a cycle that visits each node exactly once}.

Will will show HamCycle ≤p CycleSize. The idea is that a Hamiltonian cycle is just a
simple cycle that contains all of the nodes.

In order to properly deal with bad encodings, we will need a string that is not in the lan-
guage. We can construct one explicitly: let D2 be the graph with 2 vertices and no edges.
Clearly hD2 , 2i 6∈ CycleSize.

We propose the following algorithm.

F = “On input x,

1. Check that x = hGi for a graph G. If not, output hD2 , 2i.

2. Output hG, |V (G)|i.”

First, we claim F runs in polynomial time. Check the validity of the encoding in Step 1 is
certainly polynomial time. Then Step 2 takes time O(n) to count the number of vertices in G.

Next, let f be the function that F computes; we will prove correctness.

Assume x ∈ HamCycle. Then x = hGi where G is a graph with a Hamiltonian cycle.

Because G is a graph, F will pass the check in step 1. Then, because a Hamiltonian cycle
for G is a simple cycle of size |V (G)|, we have f (x) = hG, |V (G)|i ∈ CycleSize.

Assume x 6∈ HamCycle. Then either x 6= hGi or x = hGi where G is a graph with no Hamil-
tonian cycle. In the first case, then x is caught in step 1 and the output f (x) = hD2 , 2i 6∈
CycleSize. In the second case, G is a graph, so F will pass the check in Step 1. Then since a
simple cycle of size |V (G)| is a Hamiltonian cycle, we have f (x) = hG, |V (G)|i 6∈ CycleSize
(if it were, then G would have a Hamiltonian cycle).

Conclude that since HamCycle ≤p CycleSize and HamCycle is NP-hard, that Cycle-
Size is NP-hard.

18
NP Completeness

Proof Template 14: to show L is NP-Complete

1. Prove that L is in NP.

2. Prove that L is NP-Hard.

Example 9:

Claim CycleSize is NP-complete.

This follows from our proofs above that CycleSize ∈ NP and CycleSize is NP-hard.

P versus NP
We would like to think of NP-hard problems as “harder” than P problems. However, we do not
know if this is true or not! This is the famous P versus NP problem. Either way, we know that
P ⊆ NP. But some properties are dependent on the answer to this open problem.

If P = NP, then every problem1 in P is NP-complete.

If P = NP, then no NP-hard problem is in P.

NP-hard NP-hard

NP-complete

NP-complete NP
=

NP
P
=

P = NP P 6= NP

?
Figure 2: Venn diagrams of inclusions between classes depending on P = NP.

1
other than ∅ and Σ∗ . This technical restriction is by virtue of how we defined mapping reductions.

19
Appendix
Example Languages for Computability Theory
Finding the correct language to reduce to can make your proofs much quicker. Omitted entries
were not mentioned in class (you could work them out yourself as practice, but some are hard).

le
ab
le
niz
ab
le

niz

cog
ab
cog
cid

Re
co-
De

Re
Definition
ADFA Yes Yes Yes {hD, wi | D is a DFA and D accepts w}
EDFA Yes Yes Yes {hDi | D is a DFA and L(D) = ∅}
EQDFA Yes Yes Yes {hD, Ei | D, E are DFAs and L(D) = L(E)}
ACFG Yes Yes Yes {hG, wi | G is a CFG and G accepts w}
ECFG Yes Yes Yes {hGi | G is a CFG and L(G) = ∅}
EQCFG No · · {hG, Hi | G, H are CFGs and L(G) = L(H)}
ALLCFG No · · {hGi | G is a CFG and L(G) = Σ∗ }
AMBCFG No · · {hGi | G is an ambiguous CFG}
ATM No Yes No {hM, wi | M is a TM and M accepts w}
ETM No No Yes {hM i | M is a TM and L(M ) = ∅}
EQTM No No No {hM, N i | M, N are TMs and L(M ) = L(N )}
HALTTM No Yes No {hM, wi | M is a TM and M halts on w}
REGTM No No No {hM i | M is a TM and L(M ) is a regular language}
ALLTM No No No {hM i | M is a TM and L(M ) = Σ∗ }

20
Example Languages for Complexity Theory
Omitted entries are unknowns, dependent on the solution to the P versus NP problem. The
certificate listed is a reminder of how we constructed a verifier for each language. There are
other always other possible certificates or verifiers.

ard
NP

-H
P

NP
In

In
Composite Yes Yes · {hxi | x a composite number}
Certificate: a non-trivial divisor of x.
HamCycle · Yes Yes {hGi | G a graph with a cycle that visits each node exactly once}
Certificate: such a cycle.
IndSet · Yes Yes {hG, ki | G a graph with a set of k vertices, none of which are adjacent}
Certificate: such a set of vertices.
SAT · Yes Yes {hφi | φ a satisfiable Boolean formula}
Certificate: a satisfing assignment.
3SAT · Yes Yes {hφi | φ a satisfiable CNF where each clause has 3 literals}
Certificate: a satisfing assignment.
Sudoku · Yes Yes {P | P is a n2 × n2 , partially filled with integers from 1 to n2 , s.t. there exi
Certificate: a complete and valid grid.

21