15-453

FORMAL LANGUAGES,
AUTOMATA AND
COMPUTABILITY
 THURSDAY APRIL 3

REVIEW for Midterm 2

TUESDAY April 8
Definition: A Turing Machine is a 7-tuple
T = (Q, Σ, Γ, δ, q0, qaccept, qreject), where:

Q is a finite set of states

Σ is the input alphabet, where  ∉ Σ
Γ is the tape alphabet, where  ∈ Γ and Σ ⊆ Γ
δ : Q × Γ → Q × Γ × {L,R}
q0 ∈ Q is the start state
qaccept ∈ Q is the accept state
qreject ∈ Q is the reject state, and qreject ≠ qaccept
 TURING MACHINE

FINITE
STATE
q0
CONTROL

I N P U T

INFINITE TAPE
 CONFIGURATIONS

11010q700110
q7

1 1 0 1 0 0 0 1 1 0
 COMPUTATION HISTORIES
An accepting computation history is a
sequence of configurations C1,C2,…,Ck, where
1. C1 is the start configuration, C1=q0w
2. Ck is an accepting configuration, Ck=uqacceptv
3. Each Ci follows from Ci-1 via the transition
function δ
A rejecting computation history is a sequence
of configurations C1,C2,…,Ck, where
1. C1 is the start configuration,
2. Ck is a rejecting configuration, Ck=uqrejectv
3. Each Ci follows from Ci-1
 M accepts w

if and only if

there is an accepting computation

history that starts with C1=q0w
We can encode a TM as a string of 0s and 1s
start reject
n states state state

0n10m10k10s10t10r10u1…

m tape symbols blank

(first k are input accept
symbol
symbols) state

( (p,a), (q,b,L) ) = 0p10a10q10b10

NB. We assume a given convention of
describing TMs by strings in Σ*.
We may assume that any string in Σ*
describes some TM:
Either the string describes a TM by the
convention,
or if the string is gibberish at some point
then the “machine” just halts if/when a
computation gets to that point.
 A language is called Turing-recognizable or
semi-decidable or recursively enumerable
(r.e.) if some TM recognizes it
A language is called decidable or recursive
if some TM decides it

Languages semi-decidable decidable

over {0,1} (r.e.) (recursive)
languages languages
ATM = { (M,w) | M is a TM that accepts string w }
ATM is undecidable: (proof by contradiction)
Assume machine H decides ATM
Accept if M accepts w
H( (M,w) ) =
Reject if M does not accept w

Construct a new TM D as follows: on input M,

run H on (M,M) and output the opposite of H

Reject if M accepts M
D( M ) = i.e. if H(M,M) accepts
Accept if M does not accept M
i.e. if H(M,M) rejects
ATM = { (M,w) | M is a TM that accepts string w }
ATM is undecidable: (proof by contradiction)
Assume machine H decides ATM
Accept if M accepts w
H( (M,w) ) =
Reject if M does not accept w

Construct a new TM D as follows: on input M,

run H on (M,M) and output the opposite of H

Reject if M
D accepts D
M
D( M
D)=
Accept if M
D does not acceptDM
ATM = { (M,w) | M is a TM that accepts string w }
ATM is undecidable: (constructive proof & subtle)
Assume machine H SEMI-DECIDES ATM
Accept if M accepts w
H( (M,w) ) =
Rejects or Loops otherwise

Construct a new TM DH as follows:

on input M, run H on (M,M) and output the
“opposite” of H whenever possible.
 Reject if DMH acceptsDM H
(i.e. if H( DM
H , DM
H ) = Accept)

Accept ifDM H rejectsDH M

DH ( D
MH ) =
(i.e. if H(DM
H , D
MH ) = Reject)

Loops ifDM H loops onDM H

(i.e. if H( DMH
, DMH ) loops)

Note: There is no contradiction here!

DH loops on DH
We can effectively construct an instance which
does not belong to ATM (namely, (DH, DH) )
but H fails to tell us that.
 THE RECURSION THEOREM
Theorem: Let T be a Turing machine that computes
a function t : Σ* × Σ* → Σ*.
Then there is a Turing machine R that computes a
function r : Σ* → Σ*, where for every string w,

r(w) = t(<R>, w)
 THE RECURSION THEOREM
Theorem: Let T be a Turing machine that computes
a function t : Σ* × Σ* → Σ*.
Then there is a Turing machine R that computes a
function r : Σ* → Σ*, where for every string w,

r(w) = t(<R>, w)

(a,b) T t(a,b)

w R t(<R>,w)
 Recursion Theorem says:
A Turing machine can obtain its own
description (code), and compute with it
. We can use the operation:
“Obtain your own description”
in pseudocode!
Given a computable t, we can get a computable r
such that r(w) = t(<R>,w) where <R> is a description of r

INSIGHT: T (or t) is really R (or r)

 Theorem: ATM is undecidable
Proof (using the Recursion Theorem):
Assume H decides ATM (Informal Proof)
Construct machine R such that on input w:
1. Obtains its own description < R>
2. Runs H on (<R>, w) and flips the output

Running R on input w always does the

opposite of what H says it should!
 Theorem: ATM is undecidable
Proof (using the Recursion Theorem):
Assume H decides ATM (Formal Proof)
Reject if H (x, w) accepts
Let TH(x, w) =
Accept if H (x, w) rejects
(Here x is viewed as a code for a TM)
By the Recursion Theorem, there is a TM R such that:
Reject if H (<R>, w) accepts
R(w) = TH(<R>, w) =
Accept if H (<R>, w) rejects
Contradiction!
MINTM = {<M>| M is a minimal TM, wrt |<M>|}

Theorem: MINTM is not RE.

Proof (using the Recursion Theorem):
MINTM = {<M>| M is a minimal TM, wrt |<M>|}

Theorem: MINTM is not RE.

Proof (using the Recursion Theorem):
Assume E enumerates MINTM (Informal Proof)
Construct machine R such that on input w:
1. Obtains its own description <R>
2. Runs E until a machine D appears
with a longer description than of R
3. Simulate D on w

Contradiction. Why?
 MINTM = {<M>| M is a minimal TM, wrt |<M>|}

Theorem: MINTM is not RE.

Proof (using the Recursion Theorem):
Assume E enumerates MINTM (Formal Proof)

Let TE(x, w) = D(w) where <D> is first in E’s

enumeration s.t. |<D>| > |x|
By the Recursion Theorem, there is a TM R such that:
R(w) = TE(<R>, w) = D(w)
where <D> is first in E’s enumeration s.t. |<D>| > |<R>
Contradiction. Why?
 THE FIXED-POINT THEOREM
Theorem: Let f : Σ* → Σ* be a computrable
function. There is a TM R such that f(<R>)
describes a TM that is equivalent to R.

Proof: Pseudocode for the TM R:

(Informal Proof)
On input w:
1. Obtain the description <R>
2. Let g = f(<R>) and interpret g
as a code for a TM G
3. Accept w iff G(w) accepts
 THE FIXED-POINT THEOREM
Theorem: Let f : Σ* → Σ* be a computrable
function. There is a TM R such that f(<R>)
describes a TM that is equivalent to R.

Proof: Let Tf(x, w) = G(w) where <G> = f (x)

(Here f(x) is viewed as a code for a TM)

By the Recursion Theorem, there is a TM R such that:

R(w) = Tf(<R>, w) = G(w) where <G> = f (<R>)
Hence R ≡ G where <G> = f (<R>), ie <R> “≡” f (<R>)
So R is a fixed point of f !
 THE FIXED-POINT THEOREM
Theorem: Let f : Σ* → Σ* be a computrable
function. There is a TM R such that f(<R>)
describes a TM that is equivalent to R.

Example:

Suppose a virus flips the first bit of each word w

in Σ* (or in each TM).

Then there is a TM R that “remains uninfected”.

 THE RECURSION THEOREM
Theorem: Let T be a Turing machine that computes
a function t : Σ* × Σ* → Σ*.
Then there is a Turing machine R that computes a
function r : Σ* → Σ*, where for every string w,

r(w) = t(<R>, w)

(a,b) T t(a,b)

w R t(<R>,w)
 THE RECURSION THEOREM
Theorem: Let T be a Turing machine that computes
a function t : Σ* × Σ* → Σ*.
Then there is a Turing machine R that computes a
function r : Σ* → Σ*, where for every string w,

r(w) = t(<R>, w)

So first, need to show how to construct a TM

that computes its own description (ie code).
 Lemma: There is a computable function
q : Σ* → Σ*, where for any string w,
q(w) is the description (code) of a TM Pw that
on any input, prints out w and then accepts
s

w Q <Pw> Pw

w
TM Q computes q
 A TM SELF THAT PRINTS <SELF>

w’ P<B> B (<B>)
<B> B
B

B (<M>) = < P<M> M> where P<M> M (w’) = M (<M>)

So, B (<B>) = < P<B>B > where P<B>B (w’) = B (<B>)
Now, P<B>B (w’)= B(<B>) = <P<B>B >>

So, let SELF = P<B>B

 A TM SELF THAT PRINTS <SELF>

w’ P<M> M (<M>)
<M> B
M

SELF

w’ B (<B>)
<B> P<B>
w P<B> B
B
 A TM SELF THAT PRINTS <SELF>

w’ P<M>
<M> B
M

SELF

w’ P<B>
<B>
w P<B> B
B
 A NOTE ON SELF REFERENCE
Suppose in general we want to design a
program that prints its own description. How?

Print this sentence.

Print two copies of the following (the stuff = B

inside quotes), and put the second copy in
quotes:
“Print two copies of the following (the stuff = P<B>
inside quotes), and put the second copy in
quotes:”
 Let f : Σ* → Σ* be a computable function
such that w ∈ A ⇔ f(w) ∈ B

Σ* A B Σ*
f

A≤m B

Say: A is Mapping Reducible to B

Write: A ≤m B (also, ¬ A ≤m ¬ B (why?) )
 Let f : Σ* → Σ* be a computable function
such that w ∈ A ⇔ f(w) ∈ B

Σ* A B Σ*
f

A≤m B

So, if B is (semi) decidable, then so is A

(And if ¬ B is (semi) decidable, then so is ¬ A)
 ATM = { (M,w) | M is a TM that accepts string w }
HALTTM = { (M,w) | M is a TM that halts on string w }

Σ* Σ*
f
ATM ≤m HALTTM
ATM HALTTM

s ∈ Σ*

f: (M,w) → (M’, w) where M’(s) = M(s) if M(s) accepts,

Loops otherwise
So, (M, w)∈ ATM ⇔ (M’, w) ∈ HALTTM
 ATM = { (M,w) | M is a TM that accepts string w }
ETM = { M | M is a TM and L(M) = ∅ }

Σ* Σ*
f
ATM ≤m ¬ E
ATM TM ¬ ETM

s ∈ Σ*

f: (M,w) → Mw where Mw (s) = M(w) if s = w,

Loops otherwise
So, (M, w)∈ ATM ⇔ Mw ∈ ¬ ETM
 ATM = { (M,w) | M is a TM that accepts string w }
REGTM = { M | M is a TM and L(M) is regular}

Σ* Σ*
f
ATM ≤m REGTM
ATM REGTM

s ∈ Σ*
f: (M,w) → M’w where M’w (s) = accept if s = 0n1n,
M(w) otherwise
So, (M, w)∈ ATM ⇔ M’w ∈ REGTM
ETM = { M | M is a TM and L(M) = ∅ }
EQTM = {( M, N) | M, N are TMs and L(M) =L(N)}

Σ* Σ*
f
ETM ≤m EQTM
ETM EQTM

s ∈ Σ*

f: M → (M, M ∅ ) where M ∅ (s) = Loops

So, M∈ E TM ⇔ (M, M ∅ ) ∈ EQTM

 ATM = { (M,w) | M is a TM that accepts string w }
ALLPDA = { P | P is a PDA and L(P) = Σ* }

Σ* Σ*
f
ATM ≤m¬ALLPDA
ATM ¬ ALLPDA

f: (M,w) → PDA Pw where s ∈ Σ*

Pw (s) = accept iff s is NOT an accepting computation of M(w)
So, (M, w )∈ ATM ⇔ Pw ∈ ¬ ALLPDA
 ATM = { (M,w) | M is a TM that accepts string w }
FPCP = { P | P is a set of dominos with a match starting
with the first domino }

Σ* Σ*
f
ATM ≤m FPCP
ATM FPCP

Construct f: (M,w) → P(M,w) such that

(M, w )∈ ATM ⇔ P(M,w) ∈ FPCP

ATM = { (M,w) | M is a TM that accepts string w }
PCP = { P | P is a set of dominos with a match }

Σ* Σ*
f
ATM ≤m PCP
ATM PCP

Construct f: (M,w) → P(M,w) such that

(M, w )∈ ATM ⇔ P(M,w) ∈ PCP

 ATM = { (M,w) | M is a TM that accepts string w }
HALTTM = { (M,w) | M is a TM that halts on string w }
ETM = { M | M is a TM and L(M) = ∅ }
REGTM = { M | M is a TM and L(M) is regular}

EQTM = {(M, N) | M, N are TMs and L(M) =L(N)}

ALLPDA = { P | P is a PDA and L(P) = Σ* }
PCP = { P | P is a set of dominos with a match }

ALL UNDECIDABLE
Use Reductions to Prove
 ATM = { (M,w) | M is a TM that accepts string w }
HALTTM = { (M,w) | M is a TM that halts on string w }
ETM = { M | M is a TM and L(M) = ∅ } ¬ ETM

REGTM = { M | M is a TM and L(M) is regular}

EQTM = {(M, N) | M, N are TMs and L(M) =L(N)} ¬ EQTM

ALLPDA = { P | P is a PDA and L(P) = Σ* } ¬ ALLPDA
PCP = { P | P is a set of dominos with a match }

ALL UNDECIDABLE
Use Reductions to Prove
Which are SEMI-DECIDABLE?
 RICE’S THEOREM
Let L be a language over Turing machines.
Assume that L satisfies the following properties:
1. For any TMs M1 and M2, where L(M1) = L(M2),
M1 ∈ L if and only if M2 ∈ L
2. There are TMs M1 and M2,
where M1 ∈ L and M2 ∉ L

Then L is undecidable

EXTREMELY POWERFUL!
 RICE’S THEOREM
Let L be a language over Turing machines.
Assume that L satisfies the following properties:
1. For any TMs M1 and M2, where L(M1) = L(M2),
M1 ∈ L if and only if M2 ∈ L
2. There are TMs M1 and M2,
where M1 ∈ L and M2 ∉ L

Then L is undecidable
FINTM = { M | M is a TM and L(M) is
ETM = { M | M isfinite}
a TM and L(M) = ∅ }
REGTM = { M | M is a TM and L(M) is regular}
 Proof: Show L is undecidable
Show: ATM is mapping reducible to L

Σ* ATM L Σ*
f

(M,w)

(M,w)
 Proof: Show L is undecidable
Show: ATM is mapping reducible to L

Σ* ATM L Σ*
f

(M,w) M1

(M,w) M2
 RICE’S THEOREM
Proof:
Define M∅ to be a TM that never halts
Assume, WLOG, that M∅ ∉ L Why?
Let M1 ∈ L (such M1 exists, by
assumption)
Show ATM is mapping reducible to
L:
Map (M, w)  Mw where

Mw (s) = accepts if both M(w) and M1(s) accept

loops otherwise
What is the language of Mw ?
ATM is mapping reducible to L

Σ* ATM L Σ*
f

(M,w) Mw ≡ M1

(M,w) Mw ≡ M∅
QED
Corollary: The following languages are
undecidable.

ETM = { M | M is a TM and L(M) = ∅ }

REGTM = {M | M is TM and L(M) is regular}

FINTM = {M | M is a TM and L(M) is finite}

DECTM = {M | M is a TM and L(M) is
decidable}
 ORACLE TMs

Is (M,w) in
FINITE ATM?
STATE
qqYES
?
CONTROL
YES

I N P U T

INFINITE TAPE
 A Turing Reduces to B
We say A is decidable in B if there is an
oracle TM M with oracle B that decides A

A ≤T B
≤T is transitive
 ≤T VERSUS ≤m
Theorem: If A ≤m B then A ≤T B
But in general, the converse doesn’t hold!
Proof:
If A ≤m B then there is a computable function
f : Σ* → Σ*, where for every w,

w ∈ A ⇔ f(w) ∈ B
We can thus use an oracle for B to decide A

Theorem: ¬HALTTM ≤T HALTTM

Theorem: ¬HALTTM ≤m HALTTM WHY?
 THE ARITHMETIC HIERARCHY
∆ 01 = { decidable sets } (sets =
languages)
∑ 0
1
= { semi-decidable sets }

∑ 0
n+1 = { sets semi-decidable in some B ∈ ∑ 0
}
n

∆ 0n+1 = { sets decidable in some B ∈ ∑ 0

}
n

Π 0
n = { complements of sets in ∑ 0
}
n
∆ 01
∑ 01 ∆ 0
2 ∑ 0
2 ∆ 0
3
 ∑ 0 Π 0
3
3
∆ 0
3

∑ 02 Π 02
∆ 0
2

∑ 01 Π1 0

Semi- Co-semi-
decidable ∆ 0
1
decidable
Languages Languages
0
= ∑ ∩ Π
0
1 1
Decidable Languages
Definition: A decidable predicate R(x,y) is some
proposition about x and y1, where there is a TM M
such that

for all x, y, R(x,y) is TRUE ⇒ M(x,y) accepts

R(x,y) is FALSE ⇒ M(x,y) rejects

We say M “decides” the predicate R.

EXAMPLES:
R(x,y) = “x + y is less than 100”
R(<N>,y) = “N halts on y in at most 100 steps”
Kleene’s T predicate, T(<M>, x, y): M accepts x in y
steps.
1. x, y are positive integers or elements of ∑*
Theorem: A language A is semi-decidable
if and only if there is a decidable predicate R(x, y)
A = { x | ∃y R(x,y) }
such that:
Proof:
(1) If A = { x | ∃y R(x,y) } then A is semi-decidable
Because we can enumerate over all y’s

(2) If A is semi-decidable, then A = { x | ∃y R(x,y) }

Let M semi-decide A and
Let R<M>(x,y) be the Kleene T- predicate: T(<M>, x, y):
TM M accepts x in y steps (y interpreted as an integer)
R<M> is a decidable predicate (why?)
So x ∈A if and only if ∃y R<M> (x,y) is true.
Theorem

∑ 0
1
= { semi-decidable sets }
= languages of the form { x | ∃y R(x,y) }

Π 0
1
= { complements of semi-decidable sets }
= languages of the form { x | ∀y R(x,y) }

∆ 01 = { decidable sets }

= ∑ 0
1
Π
∩ 10

Where R is a decidable predicate

Theorem

∑ 0
2
= { sets semi-decidable in some semi-dec. B }
= languages of the form { x | ∃y1∀y2 R(x,y1,y2) }

Π = { complements of ∑ 2
0 0
sets}
2
= languages of the form { x | ∀y1∃y2 R(x,y1,y2) }
∆ 0
2
= ∑ 0
2
Π
∩ 20

Where R is a decidable predicate

Theorem

∑ 0
n
= languages { x | ∃y1 ∀y2 ∃y3 …Qyn R(x,y1 ,…,yn) }

Π 0
n
= languages { x | ∀y1 ∃y2 ∀y3 …Qyn R(x,y1 ,…,yn) }

∆ 0
n
= ∑ 0
n
Π
∩ n0

Where R is a decidable predicate

Example Decidable predicate

∑ 01 = languages of the form { x | ∃y R(x,y) }

We know that ATM is in∑

0
Why?
1
Show it can be described in this
form:
ATM = { <(M,w)> | ∃t [M accepts w in t steps] }

decidable predicate
ATM = { <(M,w)> | ∃t T (<M>, w, t )}
ATM = { <(M,w)> | ∃v (v is an accepting
computation history of M on w}
 ∑ 0
Π 0
3
∆0 3
3

∑ 0 Π 0
2
2

∆ 0
2
= ∑ 02 ∩ Π 02
Π 0
∑ 0
1
1
Semi- ATM Co-semi-
decidable decidable
languages ∆ 01 languages

Decidable languages
Example

Π 1 = languages of the form { x | ∀y R(x,y) }

0

Show that EMPTY (ie, ETM) = { M | L(M) = ∅ } is inΠ 0

1

EMPTY = { M | ∀w∀t [M doesn’t accept w in t steps] }

two quantifiers?? decidable predicate

Example

Π 1 = languages of the form { x | ∀y R(x,y) }

0

Show that EMPTY (ie, ETM) = { M | L(M) = ∅ } is inΠ 0

1

EMPTY = { M | ∀w∀t [ ¬T(<M>, w, t) ] }

two quantifiers?? decidable predicate

 THE PAIRING FUNCTION
Theorem. There is a 1-1 and onto computable
function < , >: Σ* x Σ* → Σ* and computable
functions π1 and π2 : Σ* → Σ* such that
z = <w, t> ⇒ π1 (z) = w and π2(z) = t

EMPTY = { M | ∀w∀t [M doesn’t accept w in t steps]

}
EMPTY = { M | ∀z[M doesn’t accept π1 (z) in π2(z) steps]}

EMPTY = { M | ∀z[ ¬T(<M>, π1 (z) , π2(z) ) ] }

 ∑ 0
Π 0
3
∆0 3
3

∑ 0 Π 0
2
2

∆ 0
2
= ∑ 02 ∩ Π 02
Π 0
∑ 0
1
1
Semi- ATM EMPTY Co-semi-
decidable decidable
languages ∆ 01 languages

Decidable languages
Example

Π 2 = languages of the form { x | ∀y∃z R(x,y,z) }

0

Show that TOTAL = { M | M halts on all inputs }

is in Π2
0

TOTAL = { M | ∀w ∃t [M halts on w in t steps] }

decidable predicate
Example

Π 2 = languages of the form { x | ∀y∃z R(x,y,z) }

0

Show that TOTAL = { M | M halts on all inputs }

is in Π2
0

TOTAL = { M | ∀w ∃t [ T(<M>, w, t) ] }

decidable predicate
 ∑ 0
Π 0
3
∆0 3
3

∑ 0 Π 0
2 TOTAL 2

∆ 0
2
= ∑ 02 ∩ Π 02
Π 0
∑ 0
1
1
Semi- ATM EMPTY Co-semi-
decidable decidable
languages ∆ 01 languages

Decidable languages
Example

∑ 2 = languages of the form { x | ∃y∀z R(x,y,z) }

0

Show that FIN = { M | L(M) is finite } is in∑ 0

2

FIN = { M | ∃n∀w∀t [Either |w| < n, or

M doesn’t accept w in t steps] }

FIN = { M | ∃n∀w∀t ( |w| < n ∨¬ T(<M>,w, t) )}

decidable predicate
 ∑ 0
Π 0
3
∆0 3
3

∑ 0
FIN Π 0
2 TOTAL 2

∆ 0
2
= ∑ 02 ∩ Π 02
Π 0
∑ 0
1
1
Semi- ATM EMPTY Co-semi-
decidable decidable
languages ∆ 01 languages

Decidable languages
 COF
∑ 0
Π 0
3
∆0 3
3

∑ 0
FIN Π 0
2 TOTAL 2

∆ 0
2
= ∑ 02 ∩ Π 02
Π 0
∑ 0
1
1
Semi- ATM EMPTY Co-semi-
decidable decidable
languages ∆ 01 languages

Decidable languages
 REG
∑ 0
Π 0
3
∆0 3
3

∑ 0
FIN Π 0
2 TOTAL 2

∆ 0
2
= ∑ 02 ∩ Π 02
Π 0
∑ 0
1
1
Semi- ATM EMPTY Co-semi-
decidable decidable
languages ∆ 01 languages

Decidable languages
 DEC
∑ 0
Π 0
3
∆0 3
3

∑ 0
FIN Π 0
2 TOTAL 2

∆ 0
2
= ∑ 02 ∩ Π 02
Π 0
∑ 0
1
1
Semi- ATM EMPTY Co-semi-
decidable decidable
languages ∆ 01 languages

Decidable languages
 CFL
∑ 0
Π 0
3
∆0 3
3

∑ 0
FIN Π 0
2 TOTAL 2

∆ 0
2
= ∑ 02 ∩ Π 02
Π 0
∑ 0
1
1
Semi- ATM EMPTY Co-semi-
decidable decidable
languages ∆ 01 languages

Decidable languages
Each is m-complete for its level in
hierachy and cannot go lower (by next
Theorem, which shows the hierarchy
does not collapse).
 ORACLES not all powerful
The following problem cannot be decided, even by
a TM with an oracle for the Halting Problem:

SUPERHALT = { (M,x) | M, with an oracle for the

Halting Problem, halts on x}

Can use diagonalization here!

Suppose H decides SUPERHALT (with oracle)
Define D(X) = “if H(X,X) accepts (with oracle)
then LOOP, else ACCEPT.”
D(D) halts ⇔ H(D,D) accepts ⇔ D(D) loops…
 ORACLES not all powerful
Theorem: The arithmetic hierarchy is strict.
That is, the nth level contains a language
that isn’t in any of the levels below n.

Proof IDEA: Same idea as the previous slide.

SUPERHALT0 = HALT = { (M,x) | M halts on x}.

SUPERHALT1 = { (M,x) | M, with an oracle for the
Halting Problem, halts on x}

SUPERHALTn = { (M,x) | M, with an oracle for

SUPERHALTn-1, halts on x}
 Theorem:
1. The hierarchy is strict
2. Each of the languages is m-complete
for its class.

Proof Idea.
1. Let ATM,1 = ATM
A TM, n+1 = {(M,x)| M is an oracle machine
with oracle ATM and M accepts x}
Then ATM, n ∈ ∑ 0
n
- Π 0
n
 Theorem:
1. The hierarchy is strict
2. Each of the languages is m-complete
for its class.

Proof.
2. Eg to show FIN is m-complete for ∑ 0
2
Need to show
FIN = { M| ∃n ∀x ∀t (|x| <n or
a) FIN ∈ ∑ 02 M does not accept x in t
steps)}
b) For A ∈ ∑ 02 then A ≤m FIN
 For A ∈ ∑
0
2 , A={ x | ∃y∀z R(x,y,z)}
FIN = { M | L(M) is finite }

Σ* Σ*
f
A FIN

Given input w:
f: x → Mx For each y of length |w| or less, look for z
such that ¬ R(x,y,z)} . If found for all such y,
Accept. Otherwise keep on running.
 For A ∈ ∑
0
2 , A={ x | ∃y∀z R(x,y,z)}
FIN = { M | L(M) is finite }

Σ* Σ*
f
A FIN

•If x ∈ A, then ∃y∀z R(x,y,z)} , so when |w| >| y|, Mx

keeps on running, so Mx∈ FIN.
•If x ∉ A, then ∀y∃z ¬ R(x,y,z)} , so Mx recognizes Σ*
 CAN WE QUANTIFY HOW MUCH
INFORMATION IS IN A STRING?

A = 01010101010101010101010101010101

B = 110010011101110101101001011001011

Idea: The more we can “compress” a string,

the less “information” it contains….
 KOLMOGOROV COMPLEXITY
Definition: Let x in {0,1}*. The shortest description
of x, denoted as d(x), is the lexicographically
shortest string <M,w> s.t. M(w) halts with x on tape.

Use pairing function to code <M,w>

Definition: The Kolmogorov complexity of x,

denoted as K(x), is |d(x)|.
 KOLMOGOROV COMPLEXITY
Theorem: There is a fixed c so that for all x in {0,1}*,
K(x) ≤ |x| + c

“The amount of information in x isn’t much more than |x|”

Proof: Define M = “On w, halt.”

On any string x, M(x) halts with x on its tape!
This implies
K(x) ≤ |<M,x>| ≤ 2|M| + |x| + 1 ≤ c + |x|
(Note: M is fixed for all x. So |M| is constant)
 REPETITIVE STRINGS

Theorem: There is a fixed c so that for all x in {0,1}*,

K(xx) ≤ K(x) + c

“The information in xx isn’t much more than that in x”

Proof: Let N = “On <M,w>, let s=M(w). Print ss.”

Let <M,w’> be the shortest description of x.
Then <N,<M,w’>> is a description of xx
Therefore
K(xx) ≤ |<N,<M,w’>>| ≤ 2|N| + K(x) + 1 ≤ c + K(x)
 REPETITIVE STRINGS

Corollary: There is a fixed c so that for all n,

and all x ∈ {0,1}*,
K(xn) ≤ K(x) + c log2 n
“The information in xn isn’t much more than that in x”
Proof:
An intuitive way to see this:
Define M: “On <x, n>, print x for n times”.
Now take <M,<x,n>> as a description of xn.
In binary, n takes O(log n) bits to write down, so we
have K(x) + O(log n) as an upper bound on K(xn).
 REPETITIVE STRINGS

Corollary: There is a fixed c so that for all n,

and all x ∈ {0,1}*,
K(xn) ≤ K(x) + c log2 n
“The information in xn isn’t much more than that in x”

Recall:
A = 01010101010101010101010101010101

For w = (01)n, K(w) ≤ K(01) + c log2 n

 CONCATENATION of STRINGS

Theorem: There is a fixed c so that for all x , y in

{0,1}*,
K(xy) ≤ 2K(x) + K(y) + c

Better: K(xy) ≤ 2 logK(x) +K(x) + K(y) + c

 INCOMPRESSIBLE STRINGS

Theorem: For all n, there is an x ∈ {0,1}n such that

K(x) ≥ n

“There are incompressible strings of every length”

Proof: (Number of binary strings of length n) = 2n

(Number of descriptions of length < n)
≤ (Number of binary strings of length < n)
= 2n – 1.

Therefore: there’s at least one n-bit string that

doesn’t have a description of length < n
 INCOMPRESSIBLE STRINGS

Theorem: For all n and c,

Prx ∈ {0,1}^n[ K(x) ≥ n-c ] ≥ 1 – 1/2c

“Most strings are fairly incompressible”

Proof: (Number of binary strings of length n) = 2n
(Number of descriptions of length < n-c)
≤ (Number of binary strings of length < n-c)
= 2n-c – 1.

So the probability that a random x has K(x) < n-c

is at most (2n-c – 1)/2n < 1/2c.
 DETERMINING COMPRESSIBILITY
Can an algorithm help us compress strings?
Can an algorithm tell us when a string is compressible?
COMPRESS = {(x,c) | K(x) ≤ c}
Theorem: COMPRESS is undecidable!

Berry Paradox: “The first string whose shortest

description cannot be written in less than fifteen
words.”
DETERMINING COMPRESSIBILITY
COMPRESS = {(x,n) | K(x) ≤ n}
Theorem: COMPRESS is undecidable!

Proof:
M = “On input x ∈ {0,1}*,
Interpret x as integer n. (|x| ≤ log n)
Find first y ∈ {0,1}* in lexicographical order,
s.t. (y,n) ∉ COMPRESS, then print y and
halt.”
M(x) prints the first string y* with K(y*) > n.
Thus <M,x> describes y*, and |<M,x>| ≤ c + log n
So n < K(y*) ≤ c + log n. CONTRADICTION!
DETERMINING COMPRESSIBILITY

Theorem: K is not computable

Proof:
M = “On input x ∈ {0,1}*,
Interpret x as integer n. (|x| ≤ log n)
Find first y ∈ {0,1}* in lexicographical order,
s. t. K(y) > n , then print y and halt.”

M(x) prints the first string y* with K(y*) > n.

Thus <M,x> describes y*, and |<M,x>| ≤ c + log n
So n < K(y*) ≤ c + log n. CONTRADICTION!
DETERMINING COMPRESSIBILITY

What about other measures of compressibility?

For example:

• the smallest DFA that recognizes {x}

• the shortest grammar in Chomsky normal form

that generates the language {x}
SO WHAT CAN YOU DO WITH THIS?
Many results in mathematics can be proved
very simply using incompressibility.
Theorem: There are infinitely many primes.
IDEA: Finitely many primes ⇒ can compress everything!

Proof: Suppose not. Let p1, … , pk be the primes.

Let x be incompressible. Think of n = x as integer.
Then there are ei s.t.
n = p1e1 … pkek
For all i, ei ≤ log n, so |ei| ≤ log log n
Can describe n (and x) with k log log n + c bits!
But x was incompressible… CONTRADICTION!
Definition: Let M be a TM that halts on all inputs.
The running time or time complexity of M is a
function f : N → N, where f(n) is the maximum
number of steps that M uses on any input of
length n.

Definition: TIME(t(n)) = { L | L is a language

decided by a O(t(n)) time Turing Machine }

P= ∪ k
TIME(n )
k∈N
Definition: A Non-Deterministic TM is a 7-tuple
T = (Q, Σ, Γ, δ, q0, qaccept, qreject), where:

Q is a finite set of states

Σ is the input alphabet, where  ∉ Σ
Γ is the tape alphabet, where  ∈ Γ and Σ ⊆ Γ
δ : Q × Γ → 2(Q × Γ × {L,R})
q0 ∈ Q is the start state
qaccept ∈ Q is the accept state
qreject ∈ Q is the reject state, and qreject ≠ qaccept
Definition: NTIME(t(n)) = { L | L is decided by a
O(t(n))-time non-deterministic Turing machine }

TIME(t(n)) ⊆ NTIME(t(n))

NP = ∪ k
NTIME(n )
k∈N
Theorem: L ∈ NP ⇔ if there exists a poly-time
Turing machine V with
L = { x | ∃y [|y| = poly(|x|) and V(x,y) accepts ] }
Proof:

(1) If L = { x | ∃y |y| = poly(|x|) and V(x,y) accepts }

then L ∈ NP
Non-deterministically guess y and then run V(x,y)
(2) If L ∈ NP then
L = { x | ∃y |y| = poly(|x|) and V(x,y) accepts }
Let N be a non-deterministic poly-time TM
that decides L, define V(x,y) to accept iff y is
an accepting computation history of N on x
A language is in NP if and only if there exist
“polynomial-length proofs’’ for membership
to the language

P = the problems that can be efficiently solved

NP = the problems where proposed solutions can
be efficiently verified

P = NP?
Can Problem Solving Be Automated?
$$$
A Clay Institute Millennium Problem
 POLY-TIME REDUCIBILITY
f : Σ* → Σ* is a polynomial time computable
function if some poly-time Turing machine M,
on every input w, halts with just f(w) on its tape

Language A is polynomial time reducible to

language B, written A ≤P B, if there is a poly-
time computable function f : Σ* → Σ* such that:

w ∈ A ⇔ f(w) ∈ B

f is called a polynomial time reduction of A to B

Theorem: If A ≤P B and B ∈ P, then A ∈ P

 SAT = { φ | (∃y)[ y is a satisfying assignment to φ
and φ is a boolean formula ] }

3SAT = { φ | (∃y)[y is a satisfying assignment to φ

and φ is in 3cnf ] }
Theorem (Cook-Levin):
SAT and 3-SAT are NP-complete

1. SAT ∈ NP:
A satisfying assignment is a “proof” that a
formula is satisfiable!

2. SAT is NP-hard:
Every language in NP can be polytime
reduced to SAT (complex formula)

Corollary: SAT ∈ P if and only if P = NP

Assume a reasonable encoding of graphs
(example: the adjacency matrix is reasonable)

CLIQUE = { (G,k) | G is an undirected graph

with a k-clique }

Theorem: CLIQUE is NP-Complete

(1) CLIQUE ∈ NP

(2) 3SAT ≤P CLIQUE

(x1 ∨ x1 ∨ x2) ∧ (¬x1 ∨ ¬x2 ∨ ¬x2) ∧ (¬x1 ∨ x2 ∨ x2)

¬x1 ¬x2 ¬x2

c x1 ¬x1
l
a
u x1 x2
s
e
x2 x2

#nodes = 3(# clauses) k = #clauses

VERTEX-COVER = { (G,k) | G is an undirected
graph with a k-node vertex cover }
Theorem: VERTEX-COVER is NP-Complete
(1) VERTEX-COVER ∈ NP
(2) 3SAT ≤P VERTEX-COVER
(x1 ∨ x1 ∨ x2) ∧ (¬x1 ∨ ¬x2 ∨ ¬x2) ∧ (¬x1 ∨ x2 ∨ x2)
Variables and negations of variables

x1 ¬x1 x2 ¬x2

clauses

x1 ¬x1 ¬x1

x1 x2 ¬x2 ¬x2 x2 x2

k = 2(#clauses) + (#variables)
HAMPATH = { (G,s,t) | G is an directed graph
with a Hamilton path from s to t}
Theorem: HAMPATH is NP-Complete
(1) HAMPATH ∈ NP
(2) 3SAT ≤P HAMPATH

Proof is in Sipser, Chapter 7.5

UHAMPATH = { (G,s,t) | G is an undirected graph
with a Hamilton path from s to t}

Theorem: UHAMPATH is NP-Complete

(1) UHAMPATH ∈ NP
(2) HAMPATH ≤P UHAMPATH
SUBSETSUM = { (S, t) | S is multiset of integers and
for some Y ⊆ S, we have ∑y∈ Y y = t }

Theorem: SUBSETSUM is NP-Complete

(1) SUBSETSUM ∈ NP
(2) 3SAT ≤P SUBSETSUM
 HW
Let G denote a graph, and s and t denote nodes.

SHORTEST PATH
= {(G, s, t, k) |
G has a simple path of length < k from s to t }

LONGEST PATH
= {(G, s, t, k) |
G has a simple path of length > k from s to t }

WHICH IS EASY? WHICH IS HARD? Justify

(see Sipser 7.21)
WWW.FLAC.WS
Good Luck on Midterm 2!