‫ر‬

‫الجامعة السعودية االلكتونية‬

‫ر‬
‫االلكتونية‬ ‫الجامعة السعودية‬

‫‪26/12/2021‬‬
Theory of Computing

Headline separator Module 10

Undecidability
Contents
1. Restricted Turing Machines
2. Turing Machines and Computers
3. A Language That Is Not Recursively Enumerable
4. An Undecidable Problem That Is RE
5. Undecidable Problems About Turing Machines
Weekly Learning Outcomes
1. Describe undecidability.
2. Explain recursively enumerable languages.
 Required Reading
1. Restricted Turing Machines
2. Turing Machines and Computers
3. Language That Is Not Recursively Enumerable
4. An Undecidable Problem That Is RE
5. Undecidable Problems About Turing Machines
(Introduction to Automata Theory, Languages, and Computation
(2013) Global Edition 3rd Edition)

Recommended Reading
https://people.cs.nctu.edu.tw/~rjchen/FormalGrad-2016/Chap-11.1.pdf

https://link.springer.com/chapter/10.1007%2F978-1-4612-1844-9_40

This Presentation is mainly dependent on the textbook: Introduction to Automata Theory, Languages, and Computation: Global Edition, 3rd edition (2013) PHI
by John E. Hopcroft, Rajeev Motwani and Jeffrey D. Ullman
• Restricted Turing Machines
Restricted Turing Machines
• Semi-infinite Tapes TM
• A TM with a semi-infinite tape means that there are no
cells to the left of the initial head position.
• A TM with a semi-infinite tape simulates a TM with an
infinite tape by using a two-track tape.

▪ The use of two tracks is as follows:

▪ The upper track represents the cells of the original TM that are
at the right of the initial head position.

▪ The lower track represents the cells at the left of the initial head
position, but in reverse order.

7
Semi-infinite Tapes TM

A semi-infinite tape that apparently can simulate a two-

way tape:
▪ Theorem 8.12

Every language accepted by a TM M2 is also accepted by

a TM M1 with the following restrictions:
⚫ M1’s head never moves left of its initial position; and
⚫ M1 never writes a blank.

8
• Turing Machines and Computers
TMs and Computers
▪ Simulating a Computer by a TM
Idea: Simulation is at the level of stored instructions and words in
memory.
▪ TM has one tape that holds all the used memory locations
and their contents.
▪ Other TM tapes hold the instruction counter, memory
address, computer input file, and “scratch.”
▪ Instruction cycle of computer simulated by:
• Find the word indicated by the instruction counter on the memory
tape.
• Examine the instruction code (a finite set of options), and get the
contents of any memory words mentioned in the instruction, using
the “scratch” tape.
• Perform the instruction, changing word values as needed, and
adding new address, value pairs to the memory tape, if needed.
• Language That Is Not Recursively Enumerable
A Language That Is Not Recursively
Enumerable

◼ A language L is recursively enumerable,

abbreviated RE if L = L(M) for some TM M.
◼ A recursive or decidable language are not only
recursively enumerable but are accepted by a TM
that always halts regardless of whether or not it
accepts.

12
Decidability vs. Undecidability
◼ There are two types of TMs (based on halting):
◼ (Recursive)
◼ TMs that always halt, no matter accepting or non-
accepting  DECIDABLE PROBLEMS
◼ (Recursively enumerable)
◼ TMs that are guaranteed to halt only on
acceptance. If non-accepting, it may or may not
halt (i.e., could loop forever).
◼ Undecidability:
◼ Undecidable problems are those that are not
recursive
13
 Recursive, RE, Undecidable languages
No TMs exist
TMs that always halt
LBA
Non-RE Languages TMs that may or
(all other languages for which may not halt
no TMs can be built)

Enumerable (RE)
Recursively
Regular Context-

sensitive
Context
(DFA)

Recursive
free
(PDA)

“Undecidable” problems
“Decidable” problems

14
Recursive Languages &
Recursively Enumerable (RE) languages
◼ Any TM for a Recursive language is going to look
like this:
“accept”
w M
“reject”

◼ Any TM for a Recursively Enumerable (RE)

language is going to look like this:
“accept”
w
M

15
 Recursive Languages are closed under
complementation

◼ If L is Recursive, L is also Recursive

M
“accept” “accept”
w
w M
“reject” “reject”

16
 Are Recursively Enumerable Languages closed
under complementation? (NO)

◼ If L is RE, L need not be RE

M
“accept” “accept” ?
w
w M
?
“reject”

17
Recursive Languages are closed under Union

◼ Let Mu = TM for L1 U L2
◼ Mu construction:
Mu
1. Make 2-tapes and copy accept
input w on both tapes M1 reject
2. Simulate M1 on tape 1 OR
3. Simulate M2 on tape 2 w accept
4. If either M1 or M2 M2 reject
accepts, then Mu
accepts
5. Otherwise, Mu rejects.

18
 Recursive Languages are closed under
Intersection
◼ Let Mn = TM for L1  L2
◼ Mn construction:
1. Make 2-tapes and copy Mn
accept
input w on both tapes M1 reject
2. Simulate M1 on tape 1 AND
AND
3. Simulate M2 on tape 2 w accept
4. If M1 AND M2 accepts, M2 reject
then Mn accepts
5. Otherwise, Mn rejects.

19
Other Closure Property Results

◼ Recursive languages are also closed under:

◼ Concatenation
◼ Kleene closure (star operator)
◼ Homomorphism, and inverse homomorphism
◼ RE languages are closed under:
◼ Union, intersection, concatenation, Kleene closure
◼ RE languages are not closed under:
◼ Complementation

20
• An Undecidable Problem That Is RE
The Diagonalization Language
The Diagonalization language

The Halting Problem

Non-RE Languages

Enumerable (RE)
Regular Context-

Recursively
sensitive
Context
(DFA)
free

Recursive
(PDA)

22
A Language about TMs & acceptance

• Let L be the language of all strings <M, w> such that:

1. M is a TM (coded in binary) with input alphabet also binary
2. w is a binary string
3. M accepts input w.

23
Enumerating all binary strings
• Let w be a binary string
• Then 1w  i, where i is some integer
• E.g., If w=, then i=1;
• If w=0, then i=2;
• If w=1, then i=3; so on…
• If 1w i, then call w as the ith word or ith binary string,
denoted by wi.
• ==> A canonical ordering of all binary strings:
• {, 0, 1, 00, 01, 10, 11, 000, 100, 101, 110, …..}
• {w1, w2, w3, w4, …. wi, … }

24
Any TM M can also be binary-coded

• M = { Q, {0, 1}, , , q0, B, F }

• Map all states, tape symbols and transitions to integers

(==>binary strings)
• (qi, Xj) = (qk, Xl, Dm) will be represented as:
• ==> 0i1 0j1 0k1 0l1 0m

• Result: Each TM can be written down as a long

binary string
• ==> Canonical ordering of TMs:
• {M1, M2, M3, M4, …. Mi, … }

25
 The Diagonalization Language

• Ld = { wi | wi  L(Mi) }
• The language of all strings whose corresponding machine does not
accept itself (i.e., its own code)
(input word w)
j • Table: T[i,j] = 1, if Mi accepts wj
= 0, otherwise.
1 2 3 4 …
(TMs) 1 0 1 0 1 …
i 2 1 1 0 0 … • Make a new language called
Ld = {wi | T[i,i] = 0}
3 0 1 0 1 …
4 1 0 0 1 …
.
.
…

.
diagonal

26
Ld is not RE (i.e., has no TM)

• Proof (by contradiction):

• Let M be the TM for Ld
• ==> M has to be equal to some Mk s.t.
L(Mk) = Ld
• ==> Will wk belong to L(Mk) or not?
1. If wk  L(Mk) ==> T[k,k]=1 ==> wk Ld
2. If wk  L(Mk) ==> T[k,k]=0 ==> wk  Ld
• A contradiction either way!!

27
Why should there be languages that do not
have TMs?

28
 Non-RE languages

How come there are languages here?

(e.g., diagonalization language)

Non-RE Languages

Enumerable (RE)
Regular Context-

Recursively
sensitive
Context
(DFA)
free

Recursive
(PDA)

29
One Explanation

There are more languages than TMs

• By pigeon hole principle:

• ==> some languages cannot have TMs

• But how do we show this?

• Need a way to “count & compare” two infinite sets

(languages and TMs)

30
How to count elements in a set?

Let A be a set:

• If A is finite ==> counting is trivial

• If A is infinite ==> how do we count?

• And, how do we compare two infinite sets by their size?

31
Cantor’s definition of set “size” for infinite
sets (1873 A.D.)
Let N = {1,2,3,…} (all natural numbers)
Let E = {2,4,6,…} (all even numbers)

Q) Which is bigger?
• A) Both sets are of the same size
• “Countably infinite”
• Proof: Show by one-to-one, onto set correspondence from
N ==> E
n f(n)
i.e, for every element in N, 1 2
there is a unique element in E, 2 4
and vice versa. 3 6
. .
. .
. .
32
Example #2
• Let Q be the set of all rational numbers
• Q = { m/n | for all m, n  N }
• Claim: Q is also countably infinite; => |Q|=|N|

1/1 1/2 1/3 1/4 1/5 ….

2/1 2/2 2/3 2/4 2/5 ….

….
3/1 3/2 3/3 3/4 3/5

4/4 4/5 ….
4/1 4/2 4/3

5/1 5/2 ….
33
 Really, really big sets!
Uncountable sets (even bigger than countably infinite sets)

Example:
• Let R be the set of all real numbers
• Claim: R is uncountable

n f(n)
1 3.14159… Build x such that x cannot possibly
2 5.55555… occur in the table
3 0.12345…
4 0.51430… E.g. x = 0 . 2 6 4 4 …
.
.
.

34
Therefore, some languages cannot have TM

• The set of all TMs is countably infinite

• The set of all Languages is uncountable

• ==> There should be some languages without TMs

35
• Undecidable Problems About Turing Machines
“Languages” vs. “Problems”

A “language” is a set of strings.

Any “problem” can be expressed as a set of all strings

that are of the form:
• “<input, output>”
e.g., Problem (a+b) ≡ Language of strings of the form { “a#b, a+b” }

==> Every problem also corresponds to a language!!

Think of the language for a “problem” == a verifier for the problem

42
The Halting Problem The Halting
Problem

An example of a recursive enumerable problem that

is also undecidable.

Non-RE Languages

Enumerable
Recursively
Regular
Context-

Contex

sensiti
(DFA)
free

Recursi

(RE)
ve
t
(PDA)

ve
43
What is the Halting Problem?

Definition of the “halting problem”:

• Does a given Turing Machine M halt on a given

input w?

Input w Machine
M

44
The Universal Turing Machine
• A Turing Machine simulator
• Given: TM M & its input w
• Aim: Build another TM called “H”, that will output:
• “accept” if M accepts w, and
• “reject” otherwise

Implies: H is in RE
• An algorithm for H:
• Simulate M on w

• H(<M, w>) =
accept, if M accepts w

reject, if M does not accept w

• Question: If M does not halt on w, what will happen to H?

45
A Claim
• Claim: No H that is always guaranteed to halt, can
exist!
• Proof: (Alan Turing, 1936)
• By contradiction, let us assume H exists

“accept”
<M,w>
H
“reject”

46
 HP Proof (step 1)
• Therefore, if H exists ➔ D also should exist.
• But can such a D exist? (if not, then H also cannot exist)
• Let us construct a new TM D using H as a
subroutine:
• On input <M>:
1. Run H on input <M, <M> >; //(i.e., run M on M
itself)
2. Output the opposite of what H outputs;

“accept” “accept”
<M>
<M, “<M>” > H
“reject” “reject”

47
HP Proof (step 2)

• The notion of inputing “<M>” to M itself

• A program can be input to itself (e.g., a compiler is a
program that takes any program as input)

accept, if M does not accept <M>

D (<M>) =
reject, if M accepts <M>

Now, what happens if D is input to itself?

accept, if D does not accept <D>

D (<D>) =
reject, if D accepts <D>

A contradiction!!! ==> Neither D nor H can exist.

48
Main Reference
1. Restricted Turing Machines
2. Turing Machines and Computers
3. Language That Is Not Recursively Enumerable
4. An Undecidable Problem That Is RE
5. Undecidable Problems About Turing Machines
(Introduction to Automata Theory, Languages, and Computation
(2013) Global Edition 3rd Edition)
Additional References
https://people.cs.nctu.edu.tw/~rjchen/FormalGrad-2016/Chap-11.1.pdf
https://link.springer.com/chapter/10.1007%2F978-1-4612-1844-9_40
https://www.cs.wmich.edu/~elise/courses/cs6800/Restricted-Machines.ppt

This Presentation is mainly dependent on the textbook: Introduction to Automata Theory, Languages, and Computation: Global Edition, 3rd edition (2013) PHI
by John E. Hopcroft, Rajeev Motwani and Jeffrey D. Ullman
Thank You