TIC 2151 – Theory of

Computation

Lecture 8
Church-Turing Thesis & Computability

TIC 2151 – Theory of Computation

1
 Lecture 8 - Outline

 Church-Turing Thesis.
 Countable sets.
 Using Gödel numbering to count Turing machines.
 Existence of non-computable problems: a simple proof.

2
 Algorithms & Turing Machines
 an algorithm is a procedure (finite sequence of instructions which can
be mechanically carried out) that terminates after a finite number of
steps for any input.
 The formal definition of algorithm emerged after the works of Alan
Turing and Alanzo Church in 1936. The church-turing thesis states that
any algorithmic procedure that can be carried out by a human (or a
computer), can also be carried out by a Turing Machine (TM).
 It is universally accepted by computer scientists that Turing machine
provides an ideal theoretical model of a computer.

3
 Algorithms & Turing Machines
Church-Turing Thesis
“a problem can be solved by an algorithm iff it can be solved by a Turing
Machine”
Notion of an algorithm

means:
= Turing machine programs

 All algorithmically solvable problems can be solved by a turing machine

 A function is computable iff it can be solved by a turing machine.
means: If we have an algorithm performing some actions to convert some
arguments x1,x2,…xn into an output y, there must be a Turing Machine TM1
program can doing it.

( ) ( )
qstart dTM1 q
€€€.. * €€€..
h(x1) h(y)
t(x1)#x2#..#xn€€.. t(y)€€..
and q is accepting 4
Algorithms & Turing Machines

Are there problems

in the world we CANNOT solve with these models?

(Unfortunately) Yes.

5
 Countable sets
A set is called countable if either it is finite OR has the same size as N.

For finite sets,. We simply count the elements in a finite set, and the resulting
number is its size. But, If we have two finite sets then they have the same size
if the elements of one set can be paired with the elements of the other set. This
method compares the sizes without resorting to counting.

if we try to count the elements of an infinite set, we will never finish! So we

can't use the counting method to determine the relative sizes of infinite sets. But
this can be done if the elements of one set can be paired with the elements of
the other set

Example:

N is countable:

M = {x | x Î N and even(x)}

Note: N and M has the same size! 6

 Countable sets
Countable sets II / Diagonalization method
m
If we let Q = { | m, n  N}
n

Note: Also Q has the same size as N.

However, for some infinite sets no correspondence with N exists. Such sets are called
uncountable. Eg: real numbers
7
 The Gödel Numbering

 A Gödel numbering for sequences provides us an effective way to

represent each finite sequence of natural numbers as a single natural
number.

 For any finite sequence x0, x1, x2… xn of natural numbers, the Gödel
number of the sequence is computed as:

Gn(x0, x1, x2… xn) = 2x03x15x2…. (PrNo(n))xn = Gödel Number

 Gn = Gödel Number function, PrNo=Prime Number.

 Example: The Gödel number of the sequence 0,2,1,0,3 is 20325170113 =
59895.
 Any number obtained this way can be uniquely factored into prime
factors, so it is possible to effectively recover the original sequence
from its Gödel number.

8
 The Gödel Numbering

 Gödel’s numbering system can be used to represent any sentence of the

formal language as a number.
 The Gödel numbering system is a one-to-one function. That simply means that it
takes in one value, which is some combination of symbols of the formal
language, and puts out another value, which is a number.

 suppose we take the sentence {x + y}, and we match numbers to the

symbols. We might define for example:
o the opening bracket symbol ‘{’ to match to 4,
o the symbol ‘x’ to match to 6,
o the symbol ‘+’ to 7,
o the symbol ‘y’ to 8,
o and the closing bracket symbol ‘}’ to 12.
 That gives you the sequence of numbers, 4, 6, 7, 8, and 12. This give us

= 3138428376721
9
 All Turing Machines and the Gödel number
We use Gödel Numbering to show that the set of all TMs is countable (if we can
represent TM using Gödel Numbering)

Now, look at the following Turing Machine:

TM T = ({qstart, q1, q2, q3, q4}, Q

{|}, S, input alphabet
{|, }, G, tape alphabet
{((qstart,)(,q1,R)), d
((q1,),(|,q2,R)),
((q1,|),( |,q1,R)),
((q2,),(,q3,L)),
((q2,|),(|,q2,R)),
((q3,|),(,q4,N))},
qstart, p0
{q4}, paccept
) preject

What was done? We only replaced Q by its set of states, q start by the start state, insert | into
10
 All Turing Machines and the Gödel number

Lets go further:

TM T = ({qstart, q1, q2, q3, q4}, {|}, {|, }, {((qstart,€)(€,q1,R)), ((q1,€),(|,q2,R)),
((q1,|),( |,q1,R)), ((q2,€),(€,q3,L)), ((q2,|),(|,q2,R)), ((q3,|),(€,q4,N))},qstart, {q4}, )

Obviously a Turing Machine program is nothing more than a word over a all the
states, the input alphabet, the tape alphabet, the stack symbols, and some special signs.

So ALL Turing Machine programs are in the set

TM  {w | w over {Q,S,G,(,),#}*}

and this set is countable!!

11
 Computing the Gödel No for TM

 Map non-numeric characters to numeric values first. Example:

Non-numeric Numeric
qstart 10
q1 12
L 13
R 14
…

 Remember Gödel represent finite sequence of natural numbers.

→There are as many TMs as there are natural numbers (countably).

12
 All C++ programs are countable

All C++ programs

The same happens to all C++ programs.

The terminals of the programming language C++ are finite. So first choose all words with
the length 1, then all the words with length 2, and so on.

Of course, a lot of words are not C++-programs, but

all the possible C++ programs are there and this set is countable

There are as many programs in a given language as there are natural numbers (countably
many).

13
 The set of all algorithms is countable
S is finite
All the algorithms are countable
S* is not finite
We’ve seen that L is the power set over
• Turing Machine programs an infinite alphabet
• C++ programs are countable sets. L cannot be countable.
There are as many programs in a given language as there are natural numbers (countably
many).

But, L = { L | L Í S*} (all the languages) See next slide

F = { f | f : N  N} //a function call more functions (all the functions)
are not countable sets.

There are as many functions on the natural numbers as there are real numbers
(uncountably many).

Hence,are noncomputable sets!!

There

there are uncountably many noncomputable functions.

14
 The set of all languages is not countable
L = {A | A  S*} is not countable.
The Proof:
 Let B be the set of all infinite binary sequences. B is uncountable.
 Let L be the set of all languages over alphabet S = {0, 1}*.
 We show that L is uncountable by giving a correspondence with B, thus showing that the
two sets are the same size.
 Let * = {s1, s2, s3…}. Each language A  L has a unique sequence in B. The i th bit of that
sequence is a 1 if si  A and otherwise 0, which is called the characteristic sequence of A
(χA). For example, if A were the language of all strings starting with a 0, we will have:
* = {, 0, 1, 00, 01, 10, 11, 000, 001, … } ;
A= { 0, 00, 01, 000, 001, … } ;
A = {0 1 0 1 1 0 0 1 1… }.
 The function f : L  B, is one-to-one and onto and hence a correspondence.Therefore, as
B is uncountable, L is uncountable as well.

This also have shown that the set of all languages cannot be put into a correspondence
with the set of all Turing machines since TM is countable. Thus we can also make a
conclusion that some languages are not recognized by any Turing machine.
15
All the problems here inside we can solve with
a Turing Machine

But problems
from here
we cannot
solve!
Lfin Í Lreg Í LDPDA Í Lcf Í Lcs Í LTM Í L

16
 Learning Outcomes

After completing this lecture, you should be able to:

 Explain what the Church-Turing thesis is (everything computable is
TM computable).
 Determine if a set is countable or not countable.
 Explain why the set of all TMs is countable (use Gödel
Numbering).
 Explain why the set of all languages (hence problems) is not
countable.
 Up to now, there is no more powerful model than TM has been
found.
 Ultimately! – Understand and explain why there exist non-
computable problems.

17