Complexity Theory

Chapter 1

Compiled By: Hussien M.

Introduction
 The notion of computation has existed in some form for thousands of
years.

 In its everyday meaning, it refers to the process of producing an output

from a set of inputs in a finite number of steps.

 Here are three examples for computational tasks:

• Given two integer numbers, compute their product.
• Given a set of n linear equations over n variables, find a solution for them
Introduction
• Computational complexity theory is concerned with how much computational resources
are required to solve a given task such as the amount of time and space.

• To solve a given problem, different models of computation were discovered such as:
• Turing machines,
• lambda calculus,
• cellular automata,
• pointer machines and
• bouncing billiard balls.

• In this course, we used Turing machine because it can be used to simulate physically
realizable computational models with very little loss of efficiency.
Introduction to TM
 Turing machine was designed by Alan Turing in 1936.
 It is a device that manipulates symbols on a strip of tape according to a set
of rules.
 Despite its simplicity, a Turing machine can be adapted to simulate the logic
of any computer algorithm, and is particularly useful in explaining the
functions of a CPU inside a computer.
 Turing machine can solve any problem that is solved by a computer.
 Turing machine also work using 0’s and 1’s
 Therefore, Turing machine is the blue print for the foundation of modern
digital computer.
 Language accepted by Turing machine: recursively enumerable language.
Introduction to TM
• Model of algorithm or computation
• A procedure that can be carried out by human beings/computer can be
carried out by a Turing machine
• It provides an ideal theoretical model of a computer.
• Turing machines are useful in several ways.
• As automaton
• As computing function
• Mathematical model for Partial Recursive function
• determining the undecidability of certain languages
• measuring the space and time complexity of problems.
Turing machine …

• Theoretical model: can model the computation of any computing

device.

• Finite control: Infinite memory in the form of a tape (all cells are
accessible). Input can be read in both directions.

• Has a designated accept state and reject state. The computation halts
when the Turing machine enters one of these two states.
 Turing Machine …
• There are various features of the Turing machine:
• It has an external memory which remembers arbitrary long sequence of input.

• It has unlimited memory capability.

• The model has a facility by which the input at left or right on the tape can be
read easily.

• The machine can produce a certain output based on its input.

• Sometimes it may be required that the same input has to be used to generate the output.

• So in this machine, the distinction between input and output has been removed.

• Thus a common set of alphabets can be used for the Turing machine.
Cont…

• In one step TM does following operations

i. a new symbol to be written on the tape in the cell under the R/W head,

ii. a motion of the R/W head along the tape: either the head moves one cell
left (L) or one cell right (R),

iii. the next state of the automaton, and

iv. whether to halt or not.

Turing machine: Language hierarchy
Cont …
Formal definition of Turing machine
• A Turing machine can be defined as a collection of 7 components:

Q: a finite nonempty set of states

∑: the finite set of input symbols
T: the tape symbol
q0: the initial state
F: a set of final states
B: a blank symbol used as a end marker for input
δ: a transition or mapping function.
Cont…
Cont…
Cont…

(ak, L)
Cont…
Cont..
Halting:
The machine halts if there are no possible transitions
Cont…
Language accepted by Turing machine

• A Turing machine (TM) accepts recursively enumerable language.

• Recursive means repeating the same set of rules for any number of times and

• Enumerable means a list of elements.

• A TM also accepts the computable functions, such as addition,

multiplication, subtraction, division, power function, and many more.
Turing Machine Example

• A Turing machine that accepts the language: aa*

Cont…
Cont…
Cont…
Cont…
Cont…
Cont…
Cont…
Cont….
Cont…
Rejection
Cont…
Cont…
Design Of Turing Machines
Guidelines:
• The fundamental objective in scanning a symbol by the R/W head is to
'know’ what to do in the future.

• The machine must remember the past symbols scanned.

• The Turing machine can remember this by going to the next unique state.

• The number of states must be minimized.

• This can be achieved by changing the states only when there is a change in
the written symbol or when there is a change in the movement of the
R/W head.
Example 1 …
Construct a Turing machine for the language ‘aba’

Solution:

• We will assume that on input tape the string 'aba' is placed as follows:

• The tape head will read out the sequence up to the Δ characters. If the tape head is
readout 'aba' string, then TM will halt after reading Δ.

• Now, we will see how this TM will work for aba. Initial state is q0 and head
points to ‘a’ as:
Example 1 …
• The move will be δ(q0, a) = δ(q1, A, R) which means it will go to
state q1, replaced ‘a’ by ‘A’ and head will move to right as:

• The move will be δ(q1, b) = δ(q2, B, R) which means it will go to

state q2, replaced ‘b’ by ‘B’ and head will move to right as:

• The move will be δ(q2, a) = δ(q3, A, R) which means it will go to

state q3, replaced ‘a’ by ‘A’ and head will move to right as:
Example 1 …
• The move δ(q3, Δ) = (q4, Δ, S) which means it will go to state q4 which is the
HALT state and HALT state is always an accept state for any TM.
• The same TM can be represented with transition table as:
Example 1 …

• The same TM can be represented with transition diagram as:

Example 2
• Construct TM for the language L ={0n1n} where n>=1.
• Solution:
• The main advantage of the TM is, it has a tape head which can be moved forward
or backward, and the input tape can be scanned.
• The simple logic which will be applied is read out each '0' mark it by A and then
move ahead along with the input tape and find out 1 convert it to B.
• Then, repeat that process for all a's and b's.
• Now we will see how this TM work for 0011.
• The simulation for 0011 can be shown as below:
Example 2…
• Now, we will see how this TM will works for 0011. Initially, state is q0 and head
points to 0 as:

• The move will be δ(q0, 0) = δ(q1, A, R) which means it will go to state q1,
replaced 0 by A and head will move to the right as:

• The move will be δ(q1, 0) = δ(q1, 0, R) which means it will not change any symbol,
remain in the same state and move to the right as:
Example 2 …
• The move will be δ(q1, 1) = δ(q2, B, L) which means it will go to state q2,
replaced 1 by B and head will move to left as:

• Now move will be δ(q2, 0) = δ(q2, 0, L) which means it will not change
any symbol, remain in the same state and move to left as:

• The move will be δ(q2, A) = δ(q0, A, R), it means will go to state q0,
replaced A by A and head will move to the right as:
Example 2…
• The move will be δ(q0, 0) = δ(q1, A, R) which means it will go to state
q1, replaced 0 by A, and head will move to right as:

• The move will be δ(q1, B) = δ(q1, B, R) which means it will not change
any symbol, remain in the same state and move to right as:

• The move will be δ(q1, 1) = δ(q2, B, L) which means it will go to state q2,
replaced 1 by B and head will move to left as:
Example 2 …
• The move δ(q2, B) = (q2, B, L) which means it will not change any symbol,
remain in the same state and move to left as:

• Now immediately before B is A that means all the 0’s are market by A. So we will
move right to ensure that no 1 is present.

• The move will be δ(q2, A) = (q0, A, R) which means it will go to state q0, will
not change any symbol, and move to right as:
Example 2 …
• The move δ(q0, B) = (q3, B, R) which means it will go to state q3, will not
change any symbol, and move to right as:

• The move δ(q3, B) = (q3, B, R) which means it will not change any symbol,
remain in the same state and move to right as:

The move δ(q3, Δ) = (q4, Δ, R) which means it will go to state q4 which is the
HALT state and HALT state is always an accept state for any TM.
Example 2 ….
Transition diagram
Cont…
Example 3
• Design a TM for Even Palindromes over the alphabet ∑ = { a, b }
• Solution: consider ababbaba
Nondeterministic Turing Machine
• Given a TM M and an input w to M, from any state Q, M can only move to a unique state
Q0. However we can also do a nondeterministic variant in which the machine can move
to multiple states simultaneously.
• Transition function: A nondeterministic TM is defined in the same way as a deterministic TM,
except for the transition function.The transition function of a nondeterministic TM is defined as

• In other words, from a single (state, symbol) pair the machine can have multiple transitions.

• States: Given a nondeterministic TM M and an input w to M, from any state Q, M can

move to multiple states in one step.

• Acceptance criterion: The machine accepts if some sequence of transitions leads to

an accept state.