Turing’s Thesis
1
Turing’s thesis:
Any computation carried out by mechanical
means can be performed by a Turing
Machine (1930)
Computer Science Law:
A computation is mechanical
if and only if
it can be performed by a Turing Machine
There is no known model of computation
more powerful than Turing Machines
2
Definition of Algorithm:
An algorithm for function f (w) is a
Turing Machine which computes f (w)
Algorithms are Turing Machines
When we say: There exists an algorithm
We mean: There exists a Turing Machine
that executes the algorithm
3
Church-Turing Thesis
• A Turing machine that halts on all inputs is
the precise formal notion corresponding to
the intuitive notion of an algorithm.
• An “algorithm” means a precisely defined
set of instructions
• This thesis cannot be formally proven
4
Consequence of Church-Turing Thesis:
If a problem cannot be solved by a Turing
machine then it cannot be solved by a human
using a precisely defined sequence of
instructions
5
Undecidability
6
– RE languages are accepted (recognized) by TM’s.
– RE languages may be grouped into two classes:
• Class 1 (recursive language) --- each language L
in this class has a TM (thought as an algorithm)
which not only accepts strings of L, but also
tells us what strings are not in L by halting.
• Class 2 (RE but not recursive) --- each language
L in this class has a TM (not thought as an
algorithm) which accepts strings of L, but may
not halt when a given input string is not in L.
7
– Formally, a language L is recursive if L = L(M)
for some TM M such that:
(1) If wL, then M accepts (and
therefore halts).
(2) If wL, then M eventually halts,
although it never enters an accepting
state (i.e., “reject” ) .
– A TM of this type corresponds to the formal
notion of algorithm.
8
A given language L, regarded as a problem,
is called decidable if L is a recursive
language; and undecidable if not.
The existence or nonexistence of an
algorithm to solve a problem (i.e., the problem
is decidable or undecidable) is more important
than the existence or nonexistence of a TM
to solve the problem.
9
Decidable Languages
Recall that:
A language A is decidable,
if there is a Turing machine M (decider)
that accepts the language A and
halts on every input string
Decision
Turing Machine M On Halt:
YES
Accept
Input
Decider for A
string NO
Reject
10
A computational problem is decidable
if the corresponding language is decidable
We also say that the problem is solvable
11
Undecidable Languages
undecidable language = not decidable language
There is no decider:
there is no Turing Machine
which accepts the language
and makes a decision (halts)
for every input string
(machine may make decision for some input strings)
12
For an undecidable language,the corresponding
problem is undecidable (unsolvable):
There is no Turing Machine (Algorithm)
that gives an answer (yes or no)
for every input instance
(answer may be given for some input instances)
13
Remember that there are undecidable
languages (i.e. also undecidable problems):
L
Turing-Acceptable L
Decidable
L is Turing-Acceptable and undecidable
14
An undecidable problem is a problem that
cannot be solved by an algorithm, regardless
of the amount of time or resources
available
We will prove that two particular problems
are unsolvable:
Membership problem
Halting problem
15
What Decidable Means
A language L is decidable if there exists a
TM M such that for all strings w:
– If w L, M enters qAccept.
– If w L, M enters qReject.
To prove a language is decidable, we can
show how to construct a TM that decides it.
For a correct proof, need a convincing argument
that the TM always eventually accepts or rejects
any input.
16
Proofs of Undecidability
To prove a language is undecidable, need to
show there is no Turing Machine that can
decide the language.
This is hard: requires reasoning about all
possible TMs.
17
Reducibility
• Method for proving that problems are
computationally unsolvable.
• A reduction is a way of converting one
problem to another problem in such a way
that a solution to the second problem can
be used to solve the first problem.
18
Reduction
Let’s say we want to solve problem of class A
and we know how to solve problems of class B
▪ If for every problem of class A, we can find a
way to convert it some problem of class B, ...
then, we can solve all problems of class A using our
method for solving problems of class B.
▪ We can also talk about how much effort is needed
to transform the problem. For most of what we
are interested in here, it is enough that the
transformation can be computed by a Turing
machine.
19
Generally…
• Reducibility plays an important role in
classifying problems by decidability, and later
in complexity theory as well.
• When A is reducible to B, solving A cannot be
harder than solving B because a solution to B
gives a solution to A.
• In terms of computability theory, if A is
reducible to B and B is decidable, A also is
decidable. Equivalently, if A is undecidable and
reducible to B, B is undecidable.
• This last version is key to proving that various
problems are undecidable.
20
Computational Complexity
Theory
21
Computational complexity theory
• Even when a problem is decidable and thus
computationally solvable in principle, it may
not be solvable in practice if the solution
requires an inordinate amount of resources
(time or space)
• And the computational complexity theory
tries to investigate the time, memory, or
other resources required for solving
computational problems.
• But, we limit our discussion to issues of
time – complexity. 22
Computational complexity theory
Goal: A general theory of the resources
needed to solve computational problems.
What types of resources?
time energy
space
What types of computational problems?
composing a poem optimization
sorting a database decision problem
23
Decision problems
A decision problem is a computational
problem with a yes or no answer.
Example: Is the number n prime?
Why focus on decision problems?
Decision problems are simple: This makes it
easy to develop a rigorous mathematical
theory.
Decision problems are surprisingly general:
Many other problems can be recast in terms
of decision problems that are essentially
equivalent. 24
Recasting other problems as decision
problems
Multiplication problem: What is the product
of m and n?
Multiplication decision problem: Is the kth
bit of the product of m and n a one?
Time required to solve one of these problems
is the same (to within a small overhead) as
the time required to solve the other.
25
Definition: A problem is easy if there is a
Turing machine to solve the problem that runs
in time polynomial in the size of the problem
input. Otherwise the problem is hard.
A problem is poly-time solvable if T(n) =O(n k)
for some constant k.
This definition is usually applied to both
decision problems and more general problems.
26
Time Complexity: The number of steps
during a computation
Space Complexity: Space used
during a computation
27
Measuring Time Complexity
• Model of computation: TM
• timereq(M) is a function of n:
• If M is deterministic
timereq(M) = f(n) = the maximum
number of steps that M executes on any
input of length n.
• If M is nondeterministic
timereq(M) = f(n) = the number of steps
on the longest path that M executes on any
input of length n.
28
Asymptotic upper bound - O
f(n) O(g(n)) iff there exists a positive
integer k and a positive constant c such
that:
n k, (f(n) c g(n)).
In other words, ignoring some number of
small cases (all those of size less than k),
and ignoring some constant factor c, f(n)
is bounded from above by g(n).
In this case, we’ll say that f is “big-oh” of g
or g asymptotically dominates f or g grows
at least as fast as f does 29
Asymptotic upper bound - O
● n3 O(n3).
● n3 O(n4).
● 3n3 O(n3).
● n3 O(3n).
● n3 O(n !).
● log n O(n).
30
Complexity Classes
31
Complexity classes: are set of languages/
functions that can be decided/ computed
within a given resource.
Now let’s introduce our first complexity
classes
Consider a deterministic Turing Machine
which decides a language
32
For any string w the computation of M
terminates in a finite amount of transitions
Initial Accept
state or Reject w
Decision Time = #transitions
Initial Accept
state or Reject w 33
Consider now all strings of length n
TM (n ) = maximum time required to decide
any string of length n
34
Time Complexity Class: TIME (T (n ))
All Languages decidable by a
deterministic Turing Machine
in time O (T (n ))
L1 L2
L3
35
Polynomial time algorithms: TIME (n )
k
constant k 0
Represents tractable algorithms:
for small k we can decide the result fast
36
The Time Complexity Class P
P = TIME (n k
)
k 0
Represents:
•polynomial time algorithms
• “tractable” problems
37
Class P
{a b }
n
{a n b n } {ww }
CYK-algorithm RELPRIME
Matrix multiplication
38
Our first computational complexity class: “P”
Definition: The set of all decision problems soluble in
polynomial time on a Turing machine is denoted P.
decision problems
P
“easy” problems
Terminology: “Multiplication is in P” “Factoring is thought not to be in P”
means “The multiplication decision means “The factoring decision
problem is in class P”. problem is thought not to be in
class P”.
39
Many important problems aren’t known to be in P
There are many problems in which a polynomial
time algorithm not known to exist to solve
them.
Example: Factoring.
Example: The traveling salesman problem (TSP).
22 km 23 km
12 km
15 km
19 km 14 km
16 km
Goal: Find the shortest tour through all the cities.
40
nk
Exponential time algorithms: TIME (2 )
Represent intractable algorithms:
Some problem instances
may take centuries to solve
41
Traveling salesman decision problem: Given a
network and a number, k, is there a tour
through all the cities of length less than k?
It is widely believed that neither of these
problems is in P.
42
Class NP
Definition: The complexity class NP consists
of all decision problems for which yes
instances of the problem have witnesses
checkable in polynomial time on a Turing
machine.
43
Examples: The factoring and TSP decision problems.
Example: The Hamiltonian cycle problem: is
there a tour that goes through each vertex in
the graph exactly once?
Many important optimization problems are in NP – it
is often hard to find a solution, but easy to verify a
solution once it is found.
44
The term NP comes from nondeterministic
polynomial time.
… and is derived from an alternative
characterization by using nondeterministic
polynomial time Turing machines.
45
The class NP
NP = NTIME (n k
)
k 0
Non-Deterministic Polynomial time
46
The P vs. NP Question
P: Languages for which membership can be
decided quickly
• Solvable by a DTM in poly-time
NP: Languages for which membership can be
verified quickly (i.e. can be tested in poly-time )
• Solvable by a NDTM in poly-time
47
Set L is in P if membership in L can be
decided in poly-time.
Set L is in NP if each x in L has a short
“proof of membership” that can be verified
in poly-time.
Fact: P NP
Question: Does NP P ?
48
The relationship of P to NP
decision problems
NP P
49
Observation:
P NP
Deterministic Non-Deterministic
Polynomial Polynomial
NP includes all problems in P and some problems
possibly outside P
Yes instances for problems in P can be
verified in poly- time, even without a witness.
50
Open Problem: P = NP ?
WE DO NOT KNOW THE ANSWER
This is the most famous unsolved problem in
computer science and mathematics.
51
Open Problem: P = NP ?
Example: Does the Satisfiability problem
have a polynomial time
deterministic algorithm?
If these classes were equal, then any
polynomally verifiable problem would be
polynomially decidable.
WE DO NOT KNOW THE ANSWER 52
Why Care?
NP Contains Lots of Problems We Don’t Know
to be in P.
Some of them are:
Hamiltonian cycle
Clique
Classroom Scheduling
Packing objects into bins
Scheduling jobs on machines
Finding cheap tours visiting a subset of cities
Allocating variables to registers
Finding good packet routings in networks
Decryption
53
We can describe just one problem L in NP,
such that if this problem L is in P,
then NP P.
It is a problem that can capture all other
problems in NP.
54
