Theory of computation

Lecture 6: Turing Machines

Adnane Saoud

Tuesday, November 21, 2023

1
 Summary of lecture 6
Lecture 6
§ Turing machines
§ Recognizers and deciders
§ Equivalence of variants of the Turing machine model
§ Church-Turing Thesis

Lecture 7
§ Notation for encodings and TMs
§ Decidability of various problems about automata and grammars:
§ Decidability for TMs.
§ Diagonalization method
§ The reducibility method
2
 Church-Turing Thesis ~1936

Algorithm = Turing
machine
Intuitive Formal

Instead of Turing machines,

can use any other “reasonable” model
Alonzo Church of unrestricted computation:
𝜆-calculus, random access machine, Alan Turing
1903–1995
1912–1954
your favorite programming language, …

3
 Hilbert’s 10th Problem
In 1900 David Hilbert posed 23 problems
#10) Give an algorithm for solving Diophantine equations.

Diophantine equations:
Equations of polynomials where solutions must be integers.
Example: 3𝑥 ! − 2𝑥𝑦 − 𝑦 ! 𝑧 = 7 solution: 𝑥 = 1, 𝑦 = 2, 𝑧 = −2

Let 𝐷 = 𝑝 polynomial 𝑝 𝑥" , 𝑥! , … , 𝑥# = 0 has a solution in integers)

Hilbert’s 10th problem: Give an algorithm to decide 𝐷.
Matiyasevich proved in 1970: 𝐷 is not decidable.

Note: 𝐷 is T-recognizable.

4
Notation for encodings

5
 Notation for encodings and TMs

Notation for encoding objects into strings

- If 𝑂 is some object (e.g., polynomial, automaton, graph, etc.),
we write 〈𝑂〉 to be an encoding of that object into a string.
- If 𝑂" , 𝑂! , … , 𝑂# is a list of objects then we write 〈𝑂" , 𝑂! , … , 𝑂# 〉
to be an encoding of them together into a single string.

Notation for writing Turing machines

We will use high-level English descriptions of algorithms when we describe TMs,
knowing that we could (in principle) convert those descriptions into states,
transition function, etc. Our notation for writing a TM 𝑀 is
𝑀 = “On input 𝑤
[English description of the algorithm]”

6
 TM – example revisited

TM 𝑀 recognizing 𝐵 = a! b! c! 𝑘 ≥ 0
𝑀 = “On input 𝑤
1. Check if 𝑤 ∈ a∗ b∗ c∗ , reject if not.
2. Count the number of a’s, b’s, and c’s in 𝑤.
3. Accept if all counts are equal; reject if not.”

High-level description is ok.

You do not need to manage tapes, states, etc…

7
Decidability of DFAs
and NFAs

8
 Acceptance Problem for DFAs

Let 𝐴DFA = 𝐵, 𝑤 𝐵 is a DFA and 𝐵 accepts 𝑤}

Theorem: 𝐴DFA is decidable
Proof: Give TM 𝐷A−DFA that decides 𝐴DFA .
𝐷A−DFA = “On input 𝑠 Shorthand:
1. Check that 𝑠 has the form 𝐵, 𝑤 where On input 𝐵, 𝑤
𝐵 is a DFA and 𝑤 is a string; reject if not.
2. Simulate the computation of 𝐵 on 𝑤.
3. If 𝐵 ends in an accept state then accept.
If not then reject.”

input tape contains 𝐵, 𝑤

𝑄 = 𝑞!, … , 𝑞" , Σ = 0,1 , 𝛿 = ⋯ , 𝑞!, 𝐹 = ⋯ , 𝑤 = 01101
𝐷A−DFA
𝐵 𝑤
𝑞& , 𝑘
work tape with current state and input head location 9
 Acceptance Problem for NFAs

Let 𝐴NFA = 𝐵, 𝑤 𝐵 is a NFA and 𝐵 accepts 𝑤}

Theorem: 𝐴NFA is decidable
Proof: Give TM 𝐷A−NFA that decides 𝐴NFA .
𝐷A−NFA = “On input 𝐵, 𝑤
1. Convert NFA 𝐵 to equivalent DFA 𝐵′.
2. Run TM 𝐷A−DFA on input 𝐵′, 𝑤 . [ Recall that 𝐷A−DFA decides 𝐴DFA ]
3. Accept if 𝐷A−DFA accepts.
Reject if not.”

New element: Use conversion construction and previously constructed TM

as a subroutine.

10
 Emptiness Problem for DFAs

Let 𝐸DFA = 𝐵 𝐵 is a DFA and 𝐿 𝐵 = ∅}

Theorem: 𝐸DFA is decidable
Proof: Give TM 𝐷E−DFA that decides 𝐸DFA .
𝐷E−DFA = “On input 𝐵 [IDEA: Check for a path from start to accept.]
1. Mark start state.
2. Repeat until no new state is marked:
Mark every state that has an incoming arrow
from a previously marked state.
3. Accept if no accept state is marked.
Reject if some accept state is marked.”

11
 Equivalence problem for DFAs

Let 𝐸𝑄DFA = { 𝐴, 𝐵 | 𝐴 and 𝐵 are DFAs and 𝐿 𝐴 = 𝐿 𝐵 }

Theorem: 𝐸𝑄DFA is decidable
Proof: Give TM 𝐷EQ−DFA that decides 𝐸𝑄DFA .
𝐷EQ−DFA = “On input 𝐴, 𝐵
[IDEA: Make DFA 𝐶 that accepts 𝑤 where 𝐴 and 𝐵 disagree.]
1. Construct DFA 𝐶 where 𝐿 𝐶 = 𝐿 𝐴 ∩ 𝐿 𝐵 ∪ 𝐿 𝐴 ∩𝐿 𝐵 .
2. Run 𝐷E−DFA on 〈𝐶〉 .
3. Accept if 𝐷E−DFA accepts.
Reject if 𝐷E−DFA rejects.”

𝐿 𝐴 𝐿 𝐵
Symmetric difference
12
 Equivalence problem for DFAs

Exercise:
Let 𝐸𝑄REX = { 𝑅" , 𝑅! | 𝑅" and 𝑅! are regular expressions and 𝐿 𝑅" = 𝐿 𝑅! }
Can we now conclude that 𝐸𝑄REX is decidable?
a) Yes, it follows immediately from things we’ve already shown.
b) Yes, but it would take significant additional work.
c) No, intersection is not a regular operation.

13
Decidability of GFGs

14
 Acceptance Problem for CFGs
Let 𝐴CFG = { 𝐺, 𝑤 | 𝐺 is a CFG and 𝑤 ∈ 𝐿 𝐺 }
Theorem: ACFG is decidable
Proof: Give TM 𝐷A−CFG that decides 𝐴CFG .
𝐷A−CFG = “On input 𝐺, 𝑤
1. Convert 𝐺 into CNF.
2. Try all derivations of length 2|𝑤| − 1.
3. Accept if any generate 𝑤.
Reject if not. Recall Chomsky Normal Form (CNF) only allows
rules:
A → BC
B→b

Lemma 1: Can convert every CFG into CNF.

Lemma 2: If 𝐻 is in CNF and 𝑤 ∈ 𝐿(𝐻) then
every derivation of 𝑤 has 2|𝑤| − 1 steps.

15
 Acceptance Problem for CFGs
Let 𝐴CFG = { 𝐺, 𝑤 | 𝐺 is a CFG and 𝑤 ∈ 𝐿 𝐺 }
Theorem: ACFG is decidable
Proof: Give TM 𝐷A−CFG that decides 𝐴CFG .
𝐷A−CFG = “On input 𝐺, 𝑤
1. Convert 𝐺 into CNF.
2. Try all derivations of length 2|𝑤| − 1.
3. Accept if any generate 𝑤.
Reject if not.

Corollary: Every CFL is decidable.

Proof: Let 𝐴 be a CFL, generated by CFG 𝐺.
Construct TM 𝑀$ = “on input 𝑤
1. Run 𝐷A−CFG on 𝐺, 𝑤 .
2. Accept if 𝐷A−CFG accepts
Reject if it rejects.”

16
 Acceptance Problem for CFGs

Exercise:
Can we conclude that 𝐴PDA is decidable?
a) Yes.
b) No, PDAs may be nondeterministic.
c) No, PDAs may not halt.

17
 Emptiness Problem for CFGs

Let 𝐸CFG = { 𝐺 | 𝐺 is a CFG and 𝐿 𝐺 = ∅ }

Theorem: 𝐸CFG is decidable
Proof:
𝐷E−CFG = “On input 𝐺 [IDEA: work backwards from terminals]
1. Mark all occurrences of terminals in 𝐺.
2. Repeat until no new variables are marked
Mark all occurrences of variable A if
A → B" B! ⋯ B# is a rule and all B% were already marked.
3. Reject if the start variable is marked.
Accept if not.”
S → R TTaa
R → Tb
T → a

18
 Equivalence Problem for CFGs

Let 𝐸𝑄CFG = { 𝐺, 𝐻 | 𝐺, 𝐻 are CFGs and 𝐿 𝐺 = 𝐿(𝐻) }

Theorem: 𝐸𝑄CFG is NOT decidable
Proof: Undecidability lecture.

Let 𝐴𝑀𝐵𝐼𝐺CFG = { 𝐺 | 𝐺 is an ambiguous CFG }

Theorem: 𝐴𝑀𝐵𝐼𝐺CFG is NOT decidable

19
 Equivalence Problem for CFGs

Exercise:
Why can’t we use the same technique we used to show 𝐸𝑄DFA is decidable
to show that 𝐸𝑄CFG is decidable?

Because CFLs are not closed under complementation and intersection.

20
Decidability of TMs

21
 Acceptance Problem for TMs

Let 𝐴TM = { 𝑀, 𝑤 | 𝑀 is a TM and 𝑀 accepts 𝑤}

Theorem: 𝐴TM is not decidable
Proof: Undecidability lecture.

Theorem: 𝐴TM is T-recognizable

Proof: The following TM 𝑈 recognizes 𝐴TM
𝑈 = “On input 𝑀, 𝑤
1. Simulate 𝑀 on input 𝑤.
2. Accept if 𝑀 halts and accepts.
3. Reject if 𝑀 halts and rejects.

Turing’s original “Universal Computing Machine”

Description of 𝑀, input 𝑤
𝑈
22
 Acceptance Problem for TMs

Let 𝐴TM = { 𝑀, 𝑤 | 𝑀 is a TM and 𝑀 accepts 𝑤}

Today’s Theorem: 𝐴TM is not decidable
Proof uses the diagonalization method,
so we will introduce that first.

23
 The Size of Infinity

How to compare the relative sizes of infinite sets?

Cantor (~1890s) had the following idea.
Definition: Say that set 𝐴 and 𝐵 have the same size if there is
bijective function 𝑓: 𝐴 → 𝐵
𝑥≠𝑦→
Range (𝑓) = 𝐵
𝑓 𝑥 ≠𝑓 𝑦
“surjective”
“injective”

Informally, two sets have the same size if we can pair up their members.
This definition works for finite sets.
Apply it to infinite sets too.

24
 Countable Sets

Let ℕ = {1,2,3, … } and let ℤ = {… , −2, −1,0,1,2, … }

Show ℕ and ℤ have the same size 𝑛 𝑓(𝑛)
ℕ 1 0 ℤ
2 -1
3 1
4 -2
5 2

Let ℚ+ = ,⁄ 6 -3
- 𝑚, 𝑛 ∈ ℕ} 7 3
Show ℕ and ℚ& have the same size ⋮ ⋮

Definition: A set is countable if it is

finite or it has the same size as ℕ.
Both ℤ and ℚ+ are countable.
25
 ℝ is Uncountable – Diagonalization

Let ℝ = all real numbers (expressible by infinite decimal expansion)

Theorem: ℝ is uncountable
Proof by contradiction via diagonalization: Assume ℝ is countable
So there is a 1-1 correspondence 𝑓: ℕ → ℝ

𝑛 𝑓(𝑛) Demonstrate a number 𝑥 ∈ ℝ that is missing from the list.

1 2.718281828…
2 3.141592653… differs from the 𝑛th number in the 𝑛th digit
3 0.000000000… so cannot be the 𝑛th number for any 𝑛.
4 1.414213562… Hence 𝑥 is not paired with any 𝑛. It is missing from the list.
5 0.142857242… Therefore 𝑓 is not a 1-1 correspondence.
6 0.207879576…
7 1.234567890…
⋮ ⋮ Diagonalization
26
 Summary

1. Notation for encodings and TMs

2. Decidability of various problems about automata and
grammars:
3. Decidability for TMs.
4. Diagonalization method

27